Time-of-check to time-of-use

TOCTOU (Time-of-check to time-of-use) is a race condition class where another process starts working on the same data between checking for any contention and the actual execution of the process.

Suppose you have a video processing worker, and multiple workers are fanned out to work on requests. To avoid race condition, a request is locked to one worker at a time. If a worker is processing requestID 1, and for some reason, another worker tries to process the same requestID, it will encounter a cache hit, which means the request is already being processed by another process.

However, another race condition can occur. What if the cache expires while the worker is still waiting for DownloadVideo step to complete, another worker tries to process the same requestID? The second worker will encounter cache miss, so it means the requestID is ready for the taking. This is where TOCTOU race condition class comes in. Literally in the name itself, this means the race condition starts between the time you check for a contention and the actual execution.

func(j Job) AddWatermark(ctx context.Context, requestID string) error {

	// ...

	// The time of check
	if err := j.cache.Get(requestID); err == nil {
		// Cache hit
		return fmt.Errorf("video locked by another process")
	} else if err != ErrCacheMiss {
		// Caching error
		return err
	}
	// Cache missed means video not locked by the process
	// We lock it now
	if err := j.cache.Set(requestID); err != nil {
		return err
	}

	// After lock, some processes happen before watermarking process
	filepath, err := j.DownloadVideo(ctx, requestID)
	if err != nil {
		return err
	}

	// The time of use

	if err := j.WatermarkVideo(ctx, filepath); err != nil {
		return err
	}

	if err := j.UploadVideo(ctx, filepath); err != nil {
		return err
	}

	if err := j.cache.Delete(requestID); err != nil {
		return err
	}

	return nil
}