So this is for writing large amounts of data to optical disks?
most likely to grab tons of discs
Oh I see, it's mostly a feeder built around an all-purpose optical drive.
Hey all, I'm curious if folks over here have played with using ipfs for archiving purposes.
Or what general interest level in something like that would be?
braxxox_: it has been brought up in the past
IPFS is not remote storage, just a way to access eomething on someone's machine
Any movement on it? @jrwr, yeah not remote storage but a great way to keep a lot of stuff safe in a lot of different places. And increase bandwidth.
This is true and has been discussed for internetarchive.bak, but the current system they have is what the community is going to stick with for now
Yeah of course :)
also IPFS has a bigger flaw that will prevent it from replacing the web for the most part
Oh yeah? What's that?
Its static, its very hard to have a webapp in IPFS
Very few users, haha
the internet is moving away from static data and more to a community
being limited to static content is both a curse *and* a blessing
if I could build a forum based community on IPFS I would like to see it being done
Because dynamic content is shit for archiving
It is and is not
IPFS + directory watchers I'm sure it could be done
A key limitation with IPFS is that, from what I understand, it requires someone to "pin" the data in order for it to actually stick around
Yep
If an organization like the IA provided public pins of archive-worthy data everyone would abuse that
true. There are just so many more moving parts with dynamic content. Plus likely nothing will run your setup in a few decades.
I would say javascript will still be around in 2050
2100, hrm, maybe
yes the language but not the web browser apis of 2017
Well, we know JS sticks around through 2035 at least ;) https://www.destroyallsoftware.com/talks/the-birth-and-death-of-javascript
@MrRadar if IA pinned content you think folks would only leech?
At least enough to ruin the bandwidth?
Overall, I don't see the web changing much in how its run in the next 50 years, Tech will change, sure but the World Wide Web will still be here
Its like IPv4, that shit should of died in 2000
Yeah I don't realistically see HTTP going anywhere fast. Everything is built on it.
The internet is like running a social network
lol true. Nobody wants to read ipv6 addys out loud haha
trying to start a new internet has a chicken and egg issue
Look at the issues IPv6 is having to even get started
braxxox_: It sucks, but a /64 or /48 is not too bad
I'm not saying ipfs will replace the web. I definitately don't think it will. but I thnk it has some amazing affordances and the distributed nature and content addressing are both super plusses for archiving.
]
I've been playing with ipfs over the past couple of days and I've got some motivation to work on setting up an archive of some sort and just seeing what happens. If nobody downloads then so be it, that would speak for itself. But any content that y'all might think would be of particular interest for such an endevor?
It works great as a kind of CDN
oops just lost my internet connection. Not sure if anybody responded to that ^. If not carry on and sry 4 tha noise.
IPFS is really just a P2P CDN
Anyway, since with archiving the data is mostly about saving the data and not serving it
presenting the data is up to the future people who will want the data.
IPFS Use case is nice for like a bittorrent but for websites
but as with turrents
if the seeds dry up, the torrent is gone
yeah that is the real danger. Especially with something as new as ipfs. Who knows how long it will be maintained?
or actively developed.
That said I don't think it is a bad idea to try. Even if at a somewhat philosophical level. Or to go through the motions of exercising the data.
between IA and textfiles
Also the best way I can think of to store ia.bak is QR codes on paper
that would last forever
Because ultimately, if everyone the web worked more like ipfs we would not have so much of the linkrot that currently crushes the web.
Ish
between IA and textfiles?
if the underlying storage goes kaboom it and another source is trying to link to it
it stills rotten
Internet Archive and Text-Files are the two biggest stores of archival computer data that I know of
Of course. But if the content it distributed and the act of accessing the data serves to dissiminate it the long tail of machines that still have it a decade later could still be very long.
This is true
Much better than our current situation. Content is on 1+ machine and all it takes is for the sysadmin to keel over and its gone for good.
overall
nothing is stopping you from trying to get people on board to do backing up things to IPFS
Yeah I hope we see a shift towards more distributed content store in the future. I think we would all be better off for it :)
Ok yeah, I'm gunna try it. I'll ping back here if I land anywhere of interest. Thanks for the mussings about :)
braxxox_: check out #internetarchive.bak
will do, thx
they are the nerds that handle the biggest project that AT covers
sweet
oh!
http://archiveteam.org/index.php?title=INTERNETARCHIVE.BAK/ipfs_implementation
looks like the IPFS devs have something in the works
oh no way!
Oh no, last modified March 2015 :(
But they at least seem like at one point they were open to it.
I don't wan't to introduce noise is all. They are up to some awesome stuff and I am sure they have themselves pretty busy.
I will bring it up over there though. IPFS has matured a lot since 2015.
greenie: Makes sense.
SketchCow: you getting the Fox Network Premiere of Wolf
recorded back on 1997-10-28
this a 125 minute movie crammed to be 2 hours with commercials
Steam Forums is starting to be uploaded.
obligatory note that IPFS is absolutely not designed for archival purposes
it's a distribution mechanism, not a storage mechanism
SketchCow: i have ~330gb of appnet sitting about that hasn't been uploaded yet. Can I push it over to FOS if your stuff is still running over there
Yes
FOS is in goodish shape at the moment.
will start sending it over now, cheers
The CD-ROM autoloader thing is working amazingly.
Except when the CD goes to shit but that'd be my life anyway
what makes me really sad is
http://ipetcompanion.com/
hours of groundbreaking virtual cat interactions unrecorded https://voatpic.me/image/6CB5BB35A2
Lord_Nigh: I'd say re-send your email to info@ with a note about it having maybe gotten lost in the cracks.
oops, ran into a non-javascript based quiz thingie on a website. wpull seems to have gotten all the combinations of all the quizzes quite well. 4 million files, 300GB D:
hahahah
wget would have OOMd early in
https://archive.org/details/tednelsonjunkmail
can i make wpull only write to a warc, not also store all the files?
Yes, use --delete-after
awesome
what does "after" refer to? immediately after downloading and parsing?
Yes, as far as I can tell
so i got the hbo airing of this: https://en.wikipedia.org/wiki/Garth:_Live_from_Central_Park
It was later edited and released on VHS, followed by its 2006 DVD release, retitled Live in Central Park, in Brooks' The Entertainer video collection.
since it said edit i think they may have change something
SketchCow: you going to Defcon this year, Always wanted to go a talk of yours
huh, didn't realise googlebots use phantomjs
I would think they would convert over to chrome
they do run chrome
just noticed while searching, maybe the pages haven't been updated in a while https://s.kurt.gg/64F8EYM.png
Interesting
hm. i read somewhere in one of their blogs years ago that they run headless chrome
the amount of CPU that must take