I remember reading litterature about that kind of video compression back in 2006. After checking it seems that H.264 allows to look back for 32 key frames, and if I remember correctly it also allows for dynamic keygrames. So a good multi-pass encoder should already be able to handle conversations with keyframes for each faces. I'm not sure about panning, but it might also be handled.I'd love a 2D cartoon animation compression. See, in cartoons, the frames stay still, while, only the mouth moves (this method is especially true in Anime, where it's mostly panning and the mouth movement is just 3 repeating frames that emulate speech). MPEG4 already did a good job in finding the differences and only sending those (ignoring the rest as "static frame", send once in a while as static keyframe), but it only looks at the previous frame, not a few frames back. Playback of that would require more memory of course, as it would need to remember many frames back (for example, a conversation between two alternating faces), and mid stream playback would be hindered too, you should only start from keyframes, but that one is just like MPEG, however, these keyframes would be further apart.