So I did some more testing and found that it happens when I have Intego Mac Security 2013 installed. I uninstalled Intego first and then installed Little Snitch and there was no hanging. On reinstalling Intego I haven't seen it hang yet.
Are there known issues when using Little Snitch with Intego security products?
Are there known issues when using Little Snitch with Intego security products?