Value based home music recording

Value based credentials
Everyone who knows me knows that I'm obsessed with getting a good value. I probably look for good deals a little too much. My wife has said to me on multiple occasions that she would really appreciate if I would stop telling everyone how much I like to get good deals. To put it bluntly, I'm a bit cheap sometimes, although I do prefer "value based" or possessing "Yankee ingenuity". The whole idea of being cheap isn't consistent with the rest of my life, since I have been known to make a fair bit of money and have spent quite a bit of it as well. The long and short of things is that I like to get the best value I can from the money I spend and I spend a fair amount of time researching things to find the right items to buy. I should probably write a separate blog just on my neurotic need to find good value, but let's just establish for now my credentials for finding good deals.  

The challenge
About 9 months ago I started the process of recording a set of songs. I'm not a particularly good musician and the other guys in our casual band are definitely not professionals either, but having set out to do something I always try to do as good a job as I can with the time and resources available.

The band
Our band is not a traditional band. We write and perform parodies for our school auction. The idea started with my good friend Jeremy and fellow school parent David who decided, off the cuff, to do a few songs during the school live auction just to be a bit different and entertaining. I joined the band the following year and it grew from there. Each fall we get together for 3 or 4 nights and write some new lyrics to classic kitschy tunes and figure out how to hack our way through them. Our lack of great talent is part of the charm of the whole affair. (Or so we like to think) I mostly think that Jeremy's jovial personality and overwhelmingly funny and positive stage presence is the only thing that makes it work. So the raw talent that we are working with is limited, but there is a fair amount of enthusiasm and positive energy that we want to capture in a musical album of the songs we have performed over the last few years.

Equipment
Knowing that I need to take the reigns and figure out how to record our band, I set out to find out how the process works and what equipment I would need to pull it off.     I'm a drummer and I have always had the additional task of managing the PA and mixing equipment.    At this point, having done it for many years, I'm pretty good at setting up microphones, speakers and mixing consoles.   I have run audio boards for other fairly decent bands in the past so I know the live performance side of things well.     Recording is a different beast and requires different tools.    Here are the things that I knew I needed:

• All the standard live equipment: Instruments, amps and microphones
• A hardware device that takes analog input from the microphones and other instruments and translates it into separate recordings of each instrument.  Each one of those is called a "track".
• A computer program that can take those separate tracks and mixes them together to create a master track that is the finished product.    Ideally this software would work similar to Photoshop and can be used to touch up and improve the tracks after they are recorded.

Microphones
Microphones are a bit like your favorite food or wine.   There is something for everyone and everyone will have a different opinion about what quality, price and sound they prefer.    Microphones are a cheap as a few dollars and can run into the thousands for bespoke models for professional studios.    If you already have a band, you probably already have some microphones.    The singer will probably care the most about what their mic sounds like.    If you don't have any mics I recommend looking at Shure, AKG, Sennheiser and Audix.    For this project I used Audix OM-5 microphones.   They are an inexpensive durable mic made for live giging.   If you are only going to do studio recording you can probably find something less durable and better sounding.   I also used a Shure wx-20 headset mic for myself (the drummer).    There are specialty mics you can get for guitar amps, bass drums, snare drums, harmonicas and every other type of instrument.     One mic I really wish I had was something like this AMT 15G for recording acoustic guitar.    A regular mic on a boom stand can work great for guitars, but the guitarist needs to have the discipline to not move around during the recording session.    A mic attached to the guitar solves those problems.    I don't really like the sound of the mics and pickups that are inside the guitar body.

Drums
Recording drums is an art unto itself.   Ideally you would mic every drum head and each cymbal separately.   Practically speaking most people just can't get there.    If you want to record an acoustic drum set properly you will need to spend more than I did on my entire recording set up.    I got around this problem by using a Roland TD-30.   High end Roland drums are pretty amazing to play, they really reproduce the feel of an acoustic set.   In live settings they are definitely a different type of instrument and don't feel quite as edgy and real, but for a recording session, they are really hard to beat, and the benefits for recording are amazing.   Everything you play will be recorded properly and if you record using MIDI you can go back and tweak the track losslessly.   You can also tailor the drum sounds to the music you are playing.    I'm a pretty weak drummer, but the TD-30 makes me sound better than I am.  (Don't worry, I'm not planning on quitting my day job)   I didn't buy the TD-30 for recording, I bought it so that I can play in my house everyday without driving my family crazy.    If you don't have an electronic drum set to record with I recommend two things.   1)  Isolate the drummer as much as possible from the rest of the band so that the drums don't bleed into the rest of the band's mics.   2) Record with a single overhead mic to capture the entire sound in one track.    You will have to play with the height to get it to sound right and the acoustics of the space you are in will color the sound, so you might have to figure out how to soften the room by hanging blankets or carpet on the walls.

Multitrack recording hardware
Your microphones have to plug into something.    At a live gig you plug all the sources into one mixing console and have someone adjust the levels of each source to sound reasonably good.     Mixing a live gig is very subjective and results vary wildly.   The nature of the live gig is that people expect to hear a more raw sound and it is often played very loudly which gives music a very different feel.   When making a recording, listeners tend to expect a more polished sound and they tend to be more critical of the recording since they listen to it over and over again.    In order to get a more polished sound it is necessary to get a clean recording of each instrument and save each one separately so that they can be mixed together into a final form later.     Traditionally, artists recorded onto tape.   8 track tapes from the 70s were are a commercialization of technology that artists used to record their individual tracks.    Today most people digitize into a computer and store each track as a file on disk.    There are lots of different products that do this.   Some are fully integrated and look like a mixing console, but also record each track to an internal hard drive.   A lot of products use FireWire to record to any computer often times a Macintosh, since they have supported FireWire for a long time.    Recent products use USB to interface with the computer and are compatible with any computer you have.  (PC, Mac or Linux)     I wanted to find a USB product since FireWire is hard to find these days and is almost impossible to get on a laptop.    The product I chose is the M-Audio C600.    The C600 can record 6 tracks simultaneously and comes from a company that has a high regard for audio quality.   There are lots of similar products out there from Tascam, Behringer, Roland, Alesis, Mackie, Motu and others.    The products are improving and changing quickly, so as you are reading this, chances are that there is or will be better products out already.   The product class is called a "USB audio interface".   Here are the basics of what you should look for:

• Number of simultaneous tracks that can be recorded.
• 1 is the minimum you can buy.   You can buy multiple single track recorders and attach them all to a computer to get multiple tracks.   This can work, but can also be a source of frustration if your computer can't handle the multiple hardware sources properly.  The cheapest ones you can buy are on Ebay for $12 each. 
• Figure out how many you tracks will need at one time.   The bigger your band the more you will need.    I figured that I would need at least 6 to mic a 4 piece band.  
• Figure out how many you can afford.   This technology is getting cheaper all the time, but there is still a pretty significant price jump from the 2 up to 6 and again up to 8 or 12 inputs.
• Overall quality.   There is a big difference in the mic pre-amps between the high and low end of the market.    Read the reviews carefully before you buy a bargain priced unit.
• Phantom power.    Transducer mics require power to operate and your audio interface should supply it.   You can plug your mics into a mixer and then key off the audio inserts on the mixer, but that represents one more step for noise to creep in.
• Rack mount or desktop.   The higher end units and especially the 8-12 input units are usually rack mount and integrate into a "professional" studio environment.    
• Headphone and audio sends.      Your audio interface will be used to provide monitoring for your band as you are recording.    The audio sends can go to speakers or you can use the headphones as a monitor.    Headphones work best since you don't want to have extra sound sources in your recording room.

Computer
Almost any reasonable computer should work.    I used a Thinkpad X220 laptop.    You will only need about 30 gigs of free disk space for most uses.    Obviously if you choose a Firewire solution, you need to have a computer that has a Firewire interface.    If you have a USB audio interface, you can use pretty much anything.    Almost everything is USB 2.0 still.   USB 3.0 is coming but isn't really necessary for most uses.

Software
Software is where you can really spend or really save, the type of software you need is call a DAW or "Digital Audio Workstation".   The 800 pound gorilla in the space is Protools.   Protools is what the pros really use and has pretty much every feature you could imagine.    Protools is expensive and has a really onerous DRM system that involves physical keys that you have to plug into your computer, it also retails for about $700 for just the base software.   There are "starter" versions of Protools and I received one with my M-Audio C600, but it turns out that the way they restrict the usage is to limit the number of simultaneous tracks you can record to two.    With only two simultaneous tracks you really can't record a band, you can only record yourself.   I'm an open source software guy and have given away nearly everything I have ever written, so I looked around to see what I could use.    I tried to use a product called Audacity.   It seemed to be the best of the open source tools, but it had a real problem interfacing with my hardware.    Most of the audio interfaces work best with a Windows driver interface called ASIO, and I couldn't make ASIO work with Audacity.    There are many DAW's out there, but I really wanted something that had a bit more polish to it and supported ASIO.   During my research I came across Reaper.   As it turns out Reaper was everything I was looking for.   It is a nicely polished product that works very reliably, supports ASIO and has very attractive pricing.    Reaper is free to try and nags you if you don't buy it.    Once you decide to buy it there are two options.   It's only $60 if you are a small time user like me, and $225 if you are a big time commercial outfit.    Most of us will fit into the $60 tier and that is a tremendous value in the DAW space.     One of the biggest benefits of Reaper, besides that fact that it works fast and reliably, is that it supports VST plugins.   VST plugins are the same things that make Protools such a powerful platform.    The DAW software is not just a recording, mixing and editing solution, it is also a platform for adding lots and lots of 3rd party effects to make your tracks sound amazing.

The recording process
Every band will do it different, but here is what I experienced.   We only had about 8 weeks from the start of the process to being completely done and we were trying to record more than 10 songs.    A real band would take longer than that just for one song in most cases.     On many of the songs we recorded the drums and rhythm guitar in one session and added the rest of the instruments as overlays later.    In some songs we tried to do all the instruments together and then went back and rerecorded some parts that we didn't like.    The complications that we ran into were many.     Sometimes one or two of our band members were not available, so we had to try and make progress.    Sometimes we would all get together and waste a bunch of time just trying to get ready.    We had problems with bleed when we tried to record multiple acoustic guitars.     Most of us didn't like our performance and wanted to go back and try it again and again.    The most important thing we did, was to keep making progress and keep making recordings.    Eventually we were able to get about 12 songs recorded in various states of completeness.   

After the band had recorded the major bits, I went back and overlayed backup vocal tracks, bongos and other percussions and started playing around with effects to try and make us sound better.

VST Plugins
After you capture your recording you will start mixing and editing.   A short while later you may  realize that you want to add effects to voices and instruments to enhance the sound.    VST plugins is where to turn for amazing effects.    Start searching the internet for VST plugins and you will find hundreds of choices for free and commercial plugins that do virtually everything.    Want to get that Autotune sound to be just like T-Pain, you can get the original commercial version and plug it right in.    Some really great free plugins are gVST and BetaBugs. 

Final cut
You may be listening to your songs on headphones and love the sound only to be surprised later on that it doesn't sound nearly as good in your car or on your iPod.    It turns out that one of the secrets of good audio engineers is that they know how to mix a song so that it sounds pretty good on all audio systems, even though it won't sound perfect on any of them.    One of the secrets is that they compress the music substantially.    Compression removes dynamic range from a song, but it also makes it sound louder overall in general.   It raises the volume of the quite parts and lowers the volume of the loud parts.   Compression is very controversial in the music world, but I will tell you that if you don't do it, people will probably tell you that your song doesn't sound right, since they will probably be listening to your song with poor quality speakers.    If your audience is all audiophiles or people with high quality headphones then leave out the compression.

Total cost and equipment list

Here is the list of equipment and approximate price for them:

• $0  A laptop or desktop computer (assumed that you already owe this) 
• $250  M-Audio C600
• $360  3 -Audix OM-5 microphone (3 x $120)
• $80    Shure WX20 headset microphone
• $200  Mic stands, cables, beer holders, etc...
• $60    Reaper DAW software

Total cost, just under $1000 USD.   The experience of recording and producing music, priceless.


The result
I know it's not terribly good, but I'm proud of the result and we had a great time making the music.   I hope these tracks are received in the spirit in which they were recorded.   I'll just post a few.

All of these songs were written for our nursery school, a COOP school in Menlo Park called the Menlo Atherton Cooperative Nursery School.   

Stop. Get Right Down and work it out
    This is a song about two toddlers battling it out on the nursery school playground.   It was inspired by a parents meeting in which we all learned the best way to try to resolve conflicts between two  children on the playground.

Can't you see what that COOP means to me
     This is the last song we wrote and is a look back at how the school had effected our lives and how special the time was.     Listening to it now brings back fond memories of all the friends I met and the great experiences helping to teach the kids in the school.

Wow, Flickr is giving away 1TB of storage for personal photo hosting

I just wrote an article for our Zetta.net blog: Cloud Storage Explained: How Can Yahoo Give Flickr Users 1TB Free?

Let me know what you think...

It's the end of <blink>!

Firefox was the last holdout of the blink tag, but has finally removed the evil.     For sentimental reasons I will show you a method for bringing it back.    Also see my other post on the creation of the blink tag.

CSS to the rescue!    Did I mention that I helped with the creation of the first version of CSS?      CSS has gotten quite a bit fancier in recent years and now supports animations, which allow you to do many very fancy tricks, one of the easiest is blinking:

<style>
.keyframeBlink {
animation-name: onOff;
animation-duration: .8s;
animation-iteration-count: infinite;
animation-direction: alternate;
}
@keyframes onOff {
    from { opacity: 1; }
    to {opacity: 0; }
}
</style>

<div class="keyframeBlink">This should blink with css animations, images work too:  </div>

This is an example of text and an image blinking!

You can also get fancier and change the rate of blinking and the style of the fade.
Here is a 1 sec duration and the use of:  "animation-timing-function: ease-in-out"

Fancy blinking example, notice the nice fade in and out!

Now kids don't try this at home, people will make fun of you. I am definitely not encouraging anyone to do this and if you ask me I will disavow any knowledge of this post.    In fact, just working on this post, which I disavow working on, is giving me a headache.



NOTES:   On chrome you will need to make the CSS a bit uglier since they are not yet supporting the new naming scheme.   They are using the -webkit- prefix on all the animation keywords.    I would expect that to change soon so that the -webkit- prefix will be unnecessary.   Here is a great site for an explanation:  http://www.css3files.com/animation/

Your style element will look like this to make it work cross browser for now:

<style>
.keyframeBlink {
-webkit-animation-name: onOff;
-webkit-animation-duration: .8s;
-webkit-animation-iteration-count: infinite;

animation-name: onOff;
animation-duration: .8s;
animation-iteration-count: infinite;
}

@keyframes onOff {
    from { opacity: 1; }
    to {opacity: 0; }
}

@-webkit-keyframes onOff {
    from { opacity: 1; }
    to {opacity: 0; }
}
</style>

Why blocking 3rd party cookies could be a bad thing

3rd party Cookies are used to facilitate targeted advertising and can be used to track your browsing across multiple Web sites.

I am not comfortable with being tracked across the web by Cookies, and Cookies were specifically designed to try and prevent this sort of behavior, so why would I be against disabling the major mechanism that is used for web tracking?   

The answer is pretty simple:

• The evil you know is better than the one you don't.
• This is probably a race we can't win.

The evil you know is better than the one you don't

Right now we know that advertising companies use 3rd party cookies to track behaviors and serve custom ads.  The companies that use these methods are well known to us and use similar methods.  They almost exclusively use 3rd party cookies.    With this knowledge it is very easy to disable tracking:  Go into your browser preferences and disable 3rd party cookies.    For those who care there is a simple and reliable method to disable tracking.   

Now lets suppose that someone decides to turn off 3rd party cookies by default in most browsers.   What do you think will happen?    Will the advertising industry just close up shop and move on to other businesses?    Will web tracking just cease to happen?    Of course not!   Advertising companies will move to other technical methods for user tracking.    They exist already, they are just a bit less convenient than 3rd party cookies to use.     The new situation is one that will be much harder for an individual user to disable tracking.  


This is probably a race we can't win

Think of this battle similar to the challenges of DRM.   Companies spend lots of time and money to design a super secure system to protect their digital content and within a week or two the DRM is broken and the content is all over the internet.    DRM is typically broken by small groups of unpaid hackers who do it for a hobby and they win just about every time.     Now think about how difficult it will be to stop a multi-billion dollar industry from winning a similar war.     If you are a browser company how are you going to stop well funded companies from coming up with technical ways to serve targeted ads?     There are already multiple ways to do tracking without 3rd party cookies and countless others could be created if enough effort is put into it.


What can we do?

My suggestion is to handle this politically not technically.   Keep working on Do-Not-Track and other mechanisms.    Keep working on legislative methods to restrict the policies.    Those are wars that we can win and those policies can have real teeth to them.     One of the benefits of the 3rd party cookie tracking techniques is that it is really easy to see who is doing the tracking.    If we make the societal choice to prohibit tracking then it will be easy to track down offenders.   


User tracking is a hot issue and should continue to be discussed and debated.    It is important to keep in mind that targeted ads are paying for almost all of the things that are on the Internet today.    If we make a wholesale decision to stop serving ads are we all prepared to start paying for all of the services that we are getting for free now?      Perhaps many people would agree to some level of targeted advertising if they saw the reasons and cost benefits clearly enough.     What I don't think will be useful is to start a technical war that disables the current working methods for disabling tracking and replaces them with poorly understood and less visible techniques.

Google "Play All" music service initial review

Today (5/15/2013) Google announced the "Play All" subscription music service.   Play All adds unlimited streaming to the Google Play Music store and app.     I have previously tried to use Google Play to store and playback music and had some problems, but I was willing to give it another go to try out this new Spotify killer service.

My history with cloud music
I am currently a Spotify premium subscriber, a Pandora premium subscriber and I have tried to use Google music and Amazon Music with limited success.    I have a very large MP3 collection that I started back in 1998 when I started ripping my own CD collection to my computer.   I have been using network based streaming music players and portable MP3 players for about 15 years.    Putting music in the cloud makes my life easier because it is available wherever I go.

The Good
The terms and signup are good: 30 days free to try it out and only $7.99 if you sign up this month.    I already had a Play account so registration was super simple.   I simply agreed to a new TOS and approved a zero dollar Google Wallet transaction.   (The same process as buying an app on the Google app store)


I started to browse around and found basically the same music selection as Spotify.    I did find some surprises when I searched on Led Zepplin and The Beatles.   In both cases I got music while Spotify is missing those artists.    As it turns out Google doesn't have them either, but music that was previously uploaded to Google Music is automatically searchable and playable.     This turns out the be the best feature of Google Music in my opinion.    All of the subscription music services are missing important artists like The Eagles, Led Zepplin, The Beatles and many others.    With Google music you can upload your own collection, up to 20,000 songs and fill in the holes that are left by the licensing problems that the music world is experiencing.    With this feature I can finally have a fully complete library with cloud convenience.

The mobile app is not as polished as Spotify is yet, but the basic features are there.   Search is good, playback is fast and I can sync and store my playlists for offline playback.    So no major problems for me, and my experience with Google apps on an Android phone has been generally very good.   I don't know how good the app is on iOS.

There is a radio feature that seems to work well.   I haven't had a chance to listen to it over a long period of time so I don't know if the algorithm is good, but time will tell.

As an experiment I started converting some of my large Spotify playlists to Google Music.  There is no automation for this, so it involves looking at the Spotify list in one window and then searching for the same song on Google Music and then adding it to a playlist.    The process is long and boring, but it did confirm that Google Music has most of the same catalog as Spotify.    There were some missing songs like "Big Yellow Taxi" by the Counting Crows but nothing really surprising or significant.   Converting playlists did bring up an important switching cost.   I have lots of playlists on Spotify and moving them will be very hard.

The Bad
I already mentioned that converting my own playlists is hard, but the other feature that I love about Spotify is that many of the people I know are also on Spotify and they share playlists with me.     Without a playlist converter I'm losing one of the best features of a socially connected music service.    This is a huge problem for Google.   If they can solve playlist sharing I can coexist peacefully with Spotify, without it I need to choose between them and the logical choice is to go where my friends are.

The other feature I'm not satisfied with is the social aspects of Google music.   Right now I can only have a private playlist or a public playlist.    I want a playlist shared with just my friends, not the whole world.   I also want collaborative playlists.

Update: I found another nit.   Google play doesn't currently work with 3rd party music players like Sonos or Squeezebox.    Since I use these in my home it's a big problem for me.    I hope that Google releases a supported API so that the service is accessible.   

Conclusion
Great pricing, the ability to upload my own music and integrate with the rest of the streaming catalog.    Overall the feature set is good to very good.     The downside to all this is your friends are probably on Spotify now and you will lose your social features if they don't move over.     The industry is still pretty young so Google has a reasonable chance to gain big market share here.    Spotify still loses money every year, so they might not even be around in a few years.  I think I will keep my subscription for a few months and see how it goes, but right now I'm not canceling Spotify.   (And if anyone from Google reads this, please create an app or plugin that converts Spotify playlists!)

The reasoning behind Web Cookies

I get a  fair number of questions about cookies from individuals and the press.    I thought I would try and explain some of the motivation and history behind Web cookies as well as some of the design behind them.      Feel free to post more questions and I will try and expand this article with more details.

See my recent post on 3rd party cookies as well!


Motivation
HTTP is the networking language behind your browser.    HTTP, or Hyper Text Transport Protocol, was designed and introduced as part of the WWW project that started with Tim Berners-Lee at CERN and was expanded upon by several universities, corporations and private individuals.    The WWW and HTTP are open standards which are published and given into the public trust in order to foster interoperability and to create products that can become ubiquitous.

One of the problems faced in the early years of the web was how to create websites that had "memory" for individual users.    The uses of "memory" on a website are many: shopping carts for shopping, personalized content, logging in, and many other interactive features require memory.    The problem in 1994 was a lack of mechanisms to identify a user individually.   HTTP was designed to be fast and efficient and part of its design was to connect to a website, grab a document and then disconnect.   This freed up the website to serve other customers, but it also meant that there was no concept of a session.     Without a session, each time a user clicked to move to a different page they would become just another random user with no way to associate them with an action they had done just moments ago.     This is a bit like talking to someone with Alzheimer disease.    Each interaction would result in having to introduce yourself again, and again, and again.

History
By the summer 1994 the idea of adding some form of "memory" to HTTP had been kicking around the WWW design groups for a while.   I'm not sure exactly how long but I would guess for 1 or 2 years.    There were some interesting proposals, but one of the popular ones that kept coming up was to add a unique identifier to every web browser so that a web site could individually identify each user use that to build a session.    I was very much against this concept because the unique identifier could be used to track a user at every website.     Without another proposal the idea of adding sessions or memory remained dormant.
 
Sometime around July of 1994 I went to a meeting to talk about a shopping server that another group at Netscape was working on.     They needed to build shopping cart functionality into the server, but they didn't have a good way to make it work with the existing technology.    They explained what they wanted to do and I had to shake my head and agree with them and tell them that they could not do it in any reasonable way with what existed.     I promised to think about the problem and get back to them.    At some point during the next week the general concept of Web Cookies formed in my head.    The idea of allowing a single web site to send a session identifier to the browser that would get sent back only to the server appealed to me and prevented cross site tracking.    I wanted to create something that had more utility so that we could do a lot more than just shopping carts, so I extended the concept of the session identifier into a general payload that would get sent back to the server.   

Design
With a rough sketch in my head I worked on the general design of cookies.    The goal was to create a session identifier and general "memory" mechanism for websites that didn't allow for cross site tracking.    I discussed the design with other members of the engineering team and got useful feedback, especially with regard to denial of service attracts and other security concerns.   John Giannandrea was especially helpful with the design.   The end result was the cookie specification that still defines 95% of what cookies do today.     A Web server can return information to the browser that the browser should give back to the server each time it contacts the server in the future.     Within the Web Cookie, a server can embed a random session identifier, a username, a shopping cart, or anything it wants as long as it doesn't try to send something too big or set too many cookies.    There are also restrictions on how cookies can be transferred, either over secure connections or not and if the cookie should be erased when the user closes the browser or reboots their computer.  


The name

I had heard the term "magic cookie" from an operating systems course from college.     The term has a somewhat similar meaning to the way Web Cookies worked and I liked the term "cookies" for aesthetic reasons.     Cookies was the first thing I came up with and the name stuck.   


Usefulness

We released the Netscape browser in the fall of 1994 and within a year it was the most popular browser in the world.     We also released the Cookie specification so that other browsers could implement it and so that web sites would know how to use it.    With most of the world using a browser that supported Web Cookies, web sites started using cookies for many different things and in ways that we could not have predicted.      Most of those uses were fantastic, some of them were concerning.

By 1996 and 1997 the web had grown dramatically and was a big business.    Most websites offered their content for free and were advertising supported.     Advertisers were looking for ways to increase the effectiveness of their ads and they looked to Web Cookies to help them with that.    


An Unforeseen Problem
A problem that I missed during the Cookie design phase was an interaction between cookies and embedded content within a webpage.    Webpages start as a single HTML document on one server.   That one HTML document contains references to other resources that are loaded to display the site that the user sees.    Images, videos, more text, and plug-ins are all references within an HTML document and any of those resources can be loaded from anywhere in the world.     This referencing technique is one of the things that make the Web so amazingly powerful.     When Web Cookies are combined with embedded references that point to other websites they are called "3rd party cookies" and they represent a new way in which users can be tracked across multiple web sites.

Advertising uses
Big ad companies, in particular DoubleClick, started using 3rd party cookies to track a browser uniquely across all of the sites that used DoubleClick to serve ads.     The use of the tracking wasn't to actually identify the name of a user, but to make sure they didn't see the same ad every time or to track the number of unique users that saw a particular ad.     Eventually the ads were customized to reflect the browsing habits of the unique identifiers.    If a browser had looked at kayaks and racquetball racquets in the past then they were given ads for sporting goods and kayaks and racquets.    

Around 1996 ad tracking via Cookies became a hot topic.    Users were upset with the practice and asking for change.     Tracking across websites was certainly not what cookies were designed to do, they were designed with the opposite intention, but what could be done at that point to fix the problem?        One of the solutions we came up with was to add controls to give each user control over what cookies to accept and from whom.    The big question at the time was whether or not to disable 3rd party cookies completely or not.     The decision had wide ranging effects since advertising was paying for most of the web and disabling 3rd party cookies would also disable many legitimate uses for cookies on embedded resources.    

An uncomfortable decision
In the end the decision to disable 3rd party cookies or keep them on was left to me.     I agonized about the decision for weeks and in the end I chose to keep them.   My logic was two fold:

Any company that had the ability to track users across a large section of the web would need to be a large publicly visible company.   Cookies could be seen by users so a tracking company can't hide from the public.    In this way the public has a natural feedback mechanism to constrain those that would seek to track them.

If 3rd party cookies were disabled ad companies would use another mechanism to accomplish the same thing, and that mechanism would not have the same level of visibility and control as cookies.   We would be trading out one problem for another.

Today, I still believe that it was the correct decision.     Governments have an ability to regulate the collection of data by large visible companies and has shown a willingness to do so.    The public has a responsibility to keep pressure on both the companies that have the ability to track users and governments to enact reasonable privacy regulations and enforce them.      Most importantly, there are other mechanisms that can replace Web cookies for tracking if they are universally disabled, and those mechanisms would be much harder to observe and disable.

Durability
The design of Cookies have remained fairly unchanged for the last 19 years.   The biggest changes have been to how users can view and control their cookies.     Many people have proposed alternatives, but none have caught on.     Cookies are not perfect, but they have certainly proved good enough and much of the functionality of the web depends on them.   

Summary
Web Cookies find themselves in the midst of a very controversial area, and have gained a level of notoriety because of it.    Even though they were designed to protect privacy they have been used in ways that are sometimes infringing upon it.    Many efforts have been taken to protect the privacy of Web users and Cookie controls have been put into the hands of every user.    The future of Cookies and user privacy is surely to continue as an interesting news item for a very long time and rightfully so.    The nature of the advertising business is to collect as much information as it possibly can, so the public needs to push back when it goes too far. 

A short history of the "about:" URL

Here is a fairly recent musing on "about:" URLs that I did just a few months ago.   

Due to my lack of a real blog this may be the first time anyone will read it. 

http://www.montulli.org/lou/about_urls

In case you are wondering, about URL's are used by your web browser to do things that URL's really should not do.

There is more info on about URLs on Wikipedia of course:  http://en.wikipedia.org/wiki/About_URI_scheme

The origins of the <blink> tag

Here is an article I posted in 2009.    I think it got picked up by a few of the tech sites as well.

http://www.montulli.org/theoriginofthe<blink>tag

Gizmodo reference: http://gizmodo.com/5903827/the-humble-origins-of-the-html-blink-tag

Recently Google named their new HTML engine after blink:  http://blog.chromium.org/2013/04/blink-rendering-engine-for-chromium.html

And I'm definitely enjoying the name of this company:  http://blinktag.com/

It seems that blink is having a resurgence in 2013, I guess it's old enough now to be retro cool.  

I'm trying out blogger

I have done a few articles and musings in the past, but I have never tried using an official blogging platform before.     A few people have asked me how to post blogs and I haven't really had a good answer, so I decided to try out Blogger for the experience.    I plan on posting links to my old content first and then tackling a few of the items on my to-do list over the next month or so.    I can then perhaps do a bit of a recursive piece on my experience with blogging on Blogger.

Fine cabinetry meets not so modern gaming

Pointer to a posting I did several years ago:  http://www.montulli.org/mame

Pictures and some story behind a MAME cabinet built in fine style.