On Fri, 28 Oct 2011 11:29:38 -0400, Faré fahree@gmail.com wrote:
Dear Christian,
I'm interested in your web scraping technology in CL.
I'd like to build a distributed web proxy that persistently records everything one views, so that you can always read and share the pages you like even when the author dies, the servers are taken off-line, the domain name is bought by someone else, and the new owner puts a new robots.txt that tells archive.org to not display the pages anymore.
I don't know if this adventure tempts you, but I think the time is ripe for end-user-controlled peer-to-peer distributed archival and sharing of information. Obvious application, beyond archival, is a distributed facebook/g+ replacement.
I cannot add anything, but express an emphatic agreement.
One important thing, IMO, would be a mathematically-sound, peer-to-peer archive authenticity co-verification -- perhaps in the same sense as git manages to do it.