You’ve seen how HTML5 gives you the first credible alternative to third-party plugins. The incompatible codec support currently makes it harder than using plugins to simply embed video in a page and have it work cross-browser.
There are already a number of ready-made scripts available that allow you to easily leverage the HTML5 synergies in your own pages, without having to do all the coding yourself. jPlayer (www.jplayer.org) is a very liberally licensed jQuery audio player that degrades to Flash in legacy browsers, can be styled with CSS, and can be extended to allow playlists. For video, you’ve already met Playr, MediaElement.js and LeanBack Player which are my weapons of choice, but many other players exist. There’s a useful video player comparison chart at http://praegnanz.de/html5video/.