Todo

-- For other people
- Multithread vp8 or vc1.
- Multithread an intra codec like mjpeg.
- Fix mpeg1 (see below).
- Try the first two items under Optimization.
- Fix h264 (see below).
- Fix mpeg4 (check mplayer too).
- Try fixing mpeg2.

-- Bug fixes

Prove correct:
- decode_update_progress() in h264.c
h264_race_checking branch has some work on h264,
but not that function. It might be worth putting
the branch under #ifdef DEBUG in mainline, but
the code would have to be cleaner.
- MPV_lowest_referenced_row() and co in mpegvideo.c
- Same in vp3.

mpeg*:
- ARM asm depends on specific offsets into MpegEncContext
which are different here.

h264:
- Files split at the wrong NAL unit don't (and can't)
be decoded with threads (e.g. TS split so PPS is after
the frame, PAFF with two fields in a packet). Scan the
packet at the start of decode and don't finish_thread_setup
until all PPS/SPS have been encountered.

mpeg4:
- Packed B-frames need to be explicitly split up
when frame threading is on. Currently disabled for
this reason.
- Support interlaced.

mpeg1:
- Seeking always prints "first frame not a keyframe"
with threads on. Currently disabled for this reason.

other:
- Keep track of and export the DTS of delayed frames, so that
clients can still use DTS for accurate a/v sync with threads.
This is required for e.g. packed B-frames to sync properly.
- Error resilience has to run before ff_report_frame_progress()
is called. Otherwise it will be unpredictable. (This might already
work.) In general testing error paths should be done more.
- 'make test' must pass with/without --disable-pthreads.
There seems to be some problem related to
draw_edges and the mpeg4 encoder.
- Add more comments to pthread.c. Rewrite the new files in doc/.

-- Optimization

- await_references() in h264 spends too much time
skipping zeros in refs[]. Add a next_array_nonzero
macro or something to use write-combining checks;
this would probably be useful somewhere else too.
- fill_edges() is disabled for h264 PAFF+MT
because it writes into the other field's
thread's pixels.
- Check update_thread_context() functions and make
sure they only copy what they need to.
- Support frame+slice threading.

-- Features

- Support streams with width/height changing. This
requires flushing all current frames (and buffering
the input in the meantime), closing the codec and
reopening it. Or don't support it.
- Support encoding. Might need more threading primitives
for good ratecontrol; would be nice for audio and libavfilter too.
- After merging to mainline, deprecate avcodec_thread_init
and just set thread_count.

-- Samples

http://astrange.ithinksw.net/ffmpeg/mt-samples/

See yuvcmp.c in this directory to compare decoded samples.

For debugging, try commenting out ff_thread_finish_setup calls so
that only one thread runs at once, and then binary search+
scatter printfs to look for differences in codec contexts.
