How do search engines index AJAX sites?

Ever wondered how web crawlers index sites whos content is generated dynamically by AJAX? Or why Twitter and Facebook now have a '#!' in their URLs?


The reason is how google works. When it sees a URL with a shabang in it, it "uglifies" the url, converting it into a slightly different one, which it then sends to the server. The server should then return a static HTML snapshot of the content of the dynamic page which the crawler indexes.


See more here: