<p>I want to be able to display a normal YouTube video with overlaid annotations, consisting of coloured rectangles for each frame. The only requirement is that this should be done programmatically. </p>

<p>YouTube has annotations now, but require you to use their front end to create them by hand. I want to be able to generate them. What's the best way of doing this?</p>

<p>Some ideas:</p>

<blockquote>
  <ol>
  <li>Build your own Flash player (ew?)</li>
  <li>Somehow draw over the YouTube Flash player. Will this work?</li>
  <li>Reverse engineer &amp; hijack YouTube's annotation system. Either messing with the local files or redirecting its attempt to download
  the annotations. (using Greasemonkey? Firefox plugin?)</li>
  </ol>
</blockquote>

<p>Idea that doesn't count: </p>

<blockquote>
  <p>download the video</p>
</blockquote>

The player itself has a
Javascript API
that might be useful for syncing the video if you choose to make your own
```
annotation-thingamajig
```
.