Wednesday, January 16, 2008

Visualization algorithm

If you've ever been to MySpace and played a song, you probably saw this:
I wouldn't say it competes with s3mp, but it is a related product. One thing I wanted to improve was the graphic equalizer. Not only does it not look good, it's just an animation--it doesn't match up with the music.

Flash actually can do better. For embedded sound, it can give you equalizer data points--enough to make a real version of what you see above. For streaming sounds, however, all it offers are left and right levels.

I implemented a visualization that has a certain resemblance to the s3mp logo. None of it was particularly hard, and I added a falloff so it looks smoother. The problem came with music like, oh, Me First and the Gimme Gimmes, where level is pegged at the top. I tried several things in different combinations, including tacking mean and standard deviation. So far, the most acceptable results display good levels, but just don't match the music right, since it's based on rolling statistics.

For now, I'm using a cubic function to remap the levels--grading on a curve, if you will. I'm not quite satisfied with the results, so I plan on trying a quintic function. With both, the goal is to make the quiet sounds louder and the loud sounds quieter, while keeping levels in order.

Technical details aside, there's an important lesson here: for those coders out there who think you won't need math, you're wrong. I used calculus and statistics for this, and couldn't have done anything without them. Sometimes, the easy way is good enough, but other times, you really need to know what you're doing.

No comments: