The Wayback Machine

What is the Wayback Machine? How can I get my site included in the Wayback Machine?

The
Internet Archive
Wayback Machine
is a service that allows people to visit archived versions of Web sites. Visitors to the Wayback Machine can type in a URL, select a date range, and then begin surfing on an archived version of the Web. Imagine surfing circa 1999 and looking at all the Y2K hype, or revisiting an older version of your favorite Web site. The Internet Archive Wayback Machine can make all of this possible.

How can I get my site included in the Wayback Machine?

Much of our archived web data comes from our own crawls or from Alexa Internet's crawls. Neither organization has a "crawl my site now!" submission process. Internet Archive's crawls tend to find sites that are well linked from other sites. The best way to ensure that we find your web site is to make sure it is included in online directories and that similar/related sites link to you.

Alexa Internet uses its own methods to discover sites to crawl. It may be helpful to install the free Alexa toolbar and visit the site you want crawled to make sure they know about it.

Regardless of who is crawling the site, you should ensure that your site's 'robots.txt' rules and in-page META robots directives do not tell crawlers to avoid your site.

When a site is crawled, there is usually at least a 6-month lag, and sometimes as much as a 24-month lag, between the date that web pages are crawled and when they appear in the Wayback Machine.

In some cases, crawled content from certain projects may appear in a much shorter timeframe — as little as a few weeks from when it was crawled. Older material for the same pages and sites may still appear separately, months later.

Yes! The Wayback Machine is built so that it can be used and referenced. If you find an archived page that you would like to reference on your Web page or in an article, you can copy the URL. You can even use fuzzy URL matching and date specification... but that's a bit more advanced.

Why isn't the site I'm looking for in the archive?

Some sites may not be included because the automated crawlers were unaware of their existence at the time of the crawl. It's also possible that some sites were not archived because they were password protected, blocked by robots.txt, or otherwise inaccessible to our automated systems. Siteowners might have also requested that their sites be excluded from the Wayback Machine. When this has occurred, you will see a "blocked site error" message. When a site is excluded because of robots.txt you will see a "robots.txt query exclusion error" message.

What does it mean when a site's archive data has been "updated"?

When our automated systems crawl the web every few months or so, we find that only about 50% of all pages on the web have changed from our previous visit. This means that much of the content in our archive is duplicate material. If you don't see ""*"" next to an archived document, then the content on the archived page is identical to the previously archived copy.

Who was involved in the creation of the Internet Archive Wayback Machine?

"The original idea for the Internet Archive Wayback Machine began in 1996, when the Internet Archive first began archiving the web. Now, five years later, with over 100 terabytes and a dozen web crawls completed, the Internet Archive has made the Internet Archive Wayback Machine available to the public. The Internet Archive has relied on donations of web crawls, technology, and expertise from Alexa Internet and others. The Internet Archive Wayback Machine is owned and operated by the Internet Archive."

How was the Wayback Machine made?

Alexa Internet, in cooperation with the Internet Archive, has designed a three dimensional index that allows browsing of web documents over multiple time periods, and turned this unique feature into the Wayback Machine.

How large is the Wayback Machine?

The Internet Archive Wayback Machine contains almost 2 petabytes of data and is currently growing at a rate of 20 terabytes per month. This eclipses the amount of text contained in the world's largest libraries, including the Library of Congress.

What type of machinery is used in this Internet Archive?

Much of the Internet Archive is stored on hundreds of slightly modified x86 servers. The computers run on the Linux operating system. Each computer has 512Mb of memory and can hold just over 1 Terabyte of data on ATA disks. However we are developing a new way of storing our data on a smaller machine. Each machine will store 1 terabyte. For more information go to www.petabox.org.

How do you archive dynamic pages?

There are many different kinds of dynamic pages, some of which are easily stored in an archive and some of which fall apart completely. When a dynamic page renders standard html, the archive works beautifully. When a dynamic page contains forms, JavaScript, or other elements that require interaction with the originating host, the archive will not contain the original site's functionality.

Why are some sites harder to archive than others?

If you look at our collection of archived sites, you will find some broken pages, missing graphics, and some sites that aren't archived at all. Here are some things that make it difficult to archive a web site:

Robots.txt -- We respect robot exclusion headers.

Javascript -- Javascript elements are often hard to archive, but especially if they generate links without having the full name in the page. Plus, if javascript needs to contact the originating server in order to work, it will fail when archived.

Server side image maps -- Like any functionality on the web, if it needs to contact the originating server in order to work, it will fail when archived.

The Standard for Robot Exclusion (SRE) is a means by which web site owners can instruct automated systems not to crawl their sites. Web site owners can specify files or directories that are disallowed from a crawl, and they can even create specific rules for different automated crawlers. All of this information is contained in a file called robots.txt. While robots.txt has been adopted as the universal standard for robot exclusion, compliance with robots.txt is strictly voluntary. In fact most web sites do not have a robots.txt file, and many web crawlers are not programmed to obey the instructions anyway. However, Alexa Internet, the company that crawls the web for the Internet Archive, does respect robots.txt instructions, and even does so retroactively. If a web site owner decides he / she prefers not to have a web crawler visiting his / her files and sets up robots.txt on the site, the Alexa crawlers will stop visiting those files and will make unavailable all files previously gathered from that site. This means that sometimes, while using the Internet Archive Wayback Machine, you may find a site that is unavailable due to robots.txt (you will see a "robots.txt query exclusion error" message). Sometimes a web site owner will contact us directly and ask us to stop crawling or archiving a site, and we endeavor to comply with these requests. When you come accross a "blocked site error" message, that means that a siteowner has made such a request and it has been honored.

Currently there is no way to exclude only a portion of a site, or to exclude archiving a site for a particular time period only.

When a URL has been excluded at direct owner request from being archived, that exclusion is retroactive and permanent.

How can I help the Internet Archive and the Wayback Machine?

The Internet Archive actively seeks donations of digital materials for preservation. If you have digital materials that may be of interest to future generations, please let us know by sending an email to info at archive dot org. The Internet Archive is also seeking additional funding to continue this important mission. You can click the donate tab above or click here. Thank you for considering us in your charitable giving.

Can I search the Archive?

Using the Internet Archive Wayback Machine, it is possible to search for the names of sites contained in the Archive (URLs) and to specify date ranges for your search. We hope to implement a full text search engine at some point in the future.

Where is the rest of the archived site? Why am I getting broken or gray images on a site?

Broken images (when there is a small red "x" where the image should be) occur when the images are not available on our servers. Usually this means that we did not archive them. Gray images are the result of robots.txt exclusions. The site in question may have blocked robot access to their images directory.

You can tell if the link you are looking for is in the Wayback Machine by entering the url into the Wayback Machine search box at archive.org (http://www.archive.org/web/web.php ). Whatever archives we have are viewable in the Wayback Machine.

The archived webpages are meant to be a “snap shot” of past Internet sites. Please note that while we try to archive an entire site, this is not always possible. That is why some images or links might be missing. Additionally some sites do not archive well and we cannot fix that. There is a list of common problems that make a site difficult to archive: http://www.archive.org/about/faqs.php#12.

If you see a box with a red X or a broken image icon that means that we unfortunately do not have the images. Files over 10MB are not archived in this “snap shot” of the website.

The best way to see all the files we have archived of the site is: http://web.archive.org/*/www.yoursite.com/*

Please note that there is a 6 - 14 month lag time between the date a site is crawled and the date it appears in the Wayback Machine.

How do I contact the Internet Archive?

All questions about the Wayback Machine, or other Internet Archive projects, should be addressed to info at archive dot org.

What is the Wayback Machine's Copyright Policy?

The Internet Archive respects the intellectual property rights and other proprietary rights of others. The Internet Archive may, in appropriate circumstances and at its discretion, remove certain content or disable access to content that appears to infringe the copyright or other intellectual property rights of others. If you believe that your copyright has been violated by material available through the Internet Archive, please provide the Internet Archive Copyright Agent with the following information:

Identification of the copyrighted work that you claim has been infringed;

An exact description of where the material about which you complain is located within the Internet Archive collections;

Your address, telephone number, and email address;

A statement by you that you have a good-faith belief that the disputed use is not authorized by the copyright owner, its agent, or the law;

A statement by you, made under penalty of perjury, that the above information in your notice is accurate and that you are the owner of the copyright interest involved or are authorized to act on behalf of that owner;

Your electronic or physical signature.

Internet Archive uses the exclusion policy intended for use by both academic and non-academic digital repositories and archivists. See our
full exclusion policy.

Why is the Internet Archive collecting sites from the Internet? What makes the information useful?

Most societies place importance on preserving artifacts of their culture and heritage. Without such artifacts, civilization has no memory and no mechanism to learn from its successes and failures. Our culture now produces more and more artifacts in digital form. The Archive's mission is to help preserve those artifacts and create an Internet library for researchers, historians, and scholars. The Archive collaborates with institutions including the Library of Congress
and the Smithsonian.

Do you archive email? Chat?

No, we do not collect or archive chat systems or personal email messages that have not been posted to Usenet bulletin boards or publicly accessible online message boards.

Do you collect all the sites on the Web?

No, we collect only publicly accessible Web pages. We do not archive pages that require a password to access, pages tagged for "robot exclusion" by their owners, pages that are only accessible when a person types into and sends a form, or pages on secure servers. If a site owner properly requests removal of a Web site through http://www.archive.org/about/exclude.php, we will exclude that site from the Wayback Machine.

Is there any personal information in these collections?

We collect Web pages that are publicly accessible. These may include pages with personal information.

Who has access to the collections? What about the public?

Anyone can access our collections through our website archive.org. The web archive can be searched using the Wayback Machine.

The Archive makes the collections available at no cost to researchers, historians, and scholars. At present, it takes someone with a certain level of technical knowledge
to access collections in a way other than our website, but there is no requirement that a user be affiliated with any particular organization.

How can I get a copy of the pages on my Web site? If my site got hacked or damaged, could I get a backup from the Archive?'

Our terms of use
do not cover backups for the general public. However, you may use the Internet Archive Wayback Machine to locate and access archived versions of a site to which you own the rights. We can't guarantee that your site has been or will be archived. We can no longer offer the service to pack up sites that have been lost. We recommend using the Warrick Tool. Please keep in mind that this is a third party and we can not promise results.

Can people download sites from the Wayback?

Our terms of use
specify that users of the Wayback Machine are not to copy data from the collection.

How do you protect my privacy if you archive my site?

The Archive collects Web pages that are publicly available the same ones that you might find as you surfed around the Web. We do not archive pages that require a password to access, pages tagged for "robot exclusion" by their owners, pages that are only accessible when a person types into and sends a form, or pages on secure servers. We also provide information on removing a site from the collections. Those who use the collections must agree to certain terms of use.

Like a public library, the Archive provides free and open access to its collections to researchers, historians, and scholars. Our cultural norms have long promoted access to documents that were, but no longer are, publicly accessible.

Given the rate at which the Internet is changing the average life of a Web page is only 77 days if no effort is made to preserve it, it will be entirely and irretrievably lost. Rather than let this moment slip by, we are proceeding with documenting the growth and content of the Internet, using libraries as our model.

Below is a list of the main error messages you will see while searching the Wayback Machine. If you see an error message that does not have the Internet Archive Wayback Machine logo in the upper left corner, you are most likely looking at an archived page or the live web.

Failed Connection: The server that the particular piece of information lives on is down. Generally these clear up within two weeks.

Robots.txt Query Exclusion: A robots.txt is something that a site owner puts on their site that keeps crawlers like our own from crawling them. The Internet Archive retroactively respects all robots.txt.

Blocked Site Error: Site owners, copyright holders and others who fit Internet Archive's exclusion policy have requested that the site be excluded from the Wayback Machine. For exclusion criteria, please see our exclusion policy
(we use the same one used and developed by other digital repositories and archivists both academic and non-academic).

Path Index Error: A path index error message refers to a problem in our database wherein the information requested is not available (generally because of a machine or software issue, however each case can be different). We cannot always completely fix these errors in a timely manner.

Not in Archive: Generally this means that the site archived has a redirect on it and the site you are redirected to is not in the archive or cannot be found on the live web.

Why are there no recent archives in the Wayback Machine?

It generally takes 6 months or more (up to 24 months) for pages to appear in the Wayback Machine after they are collected, because of delays in transferring material to long-term storage and indexing, or the requirements of our collection partners.

In some cases, crawled content from certain projects can appear in a much shorter timeframe — as little as a few weeks from when it was crawled. Older material for the same pages and sites may still appear separately, months later.

There is no access to files before they appear in the Wayback Machine.

How does the Wayback Machine behave with Javascript turned off?

If you have Javascript turned off, images and links will be from the live web, not from our archive of old Web files.

How did I end up on the live version of a site? or I clicked on X date, but now I am on Y date, how is that possible? Why can I only see 930 out of the 2000 results?

How did I end up on the live version of a site? or I clicked on X date, but now I am on Y date, how is that possible?

Not every date for every site archived is 100% complete. When you are surfing an incomplete archived site the Wayback Machine will grab the closest available date to the one you are in for the links that are missing. In the event that we do not have the link archived at all, the Wayback Machine will look for the link on the live web and grab it if available. Pay attention to the date code embedded in the archived url. This is the list of numbers in the middle; it translates as yyyymmddhhmmss. For example in this url http://web.archive.org/web/20000229123340/http://www.yahoo.com/ the date the site was crawled was Feb 29, 2000 at 12:33 and 40 seconds.

You can see a listing of the dates of the specific URL by replacing the date code with an asterisk (*), ie: http://web.archive.org/*/www.yoursite.com

Whatever archives we have are viewable in the Wayback Machine. Please note that there is a 6 - 14 month lag time between the date a site is crawled and the date it appears in the Wayback Machine.

Why can I only see 930 out of the 2000 results?

The list of results displayed shows the total number of pages we have for a given domain name. This includes numerous repeats as we return to sites to recrawl their content. The reported results is this total; whereas the smaller number relates to the number of unique results only.

Where does the name come from?

The Wayback Machine is named in reference to the famous Mr. Peabody's WABAC (pronounced way-back) machine from the Rocky and Bullwinkle cartoon show.

How do I cite Wayback Machine urls in MLA format?

This question is a newer one. We asked MLA to help us with how to cite an archived URL in correct format. They did say that there is no established format for resources like the Wayback Machine, but it's best to err on the side of more information. You should cite the webpage as you would normally, and then give the Wayback Machine information. They provided the following example: McDonald, R. C. "Basic Canary Care." _Robirda Online_. 12 Sept. 2004. 18 Dec. 2006 . _Internet Archive_. < http://web.archive.org/web/20041009202820/http://www.robirda.com/cancare.html>. They added that if the date that the information was updated is missing, one can use the closest date in the Wayback Machine. Then comes the date when the page is retrieved and the original URL. Neither URL should be underlined in the bibliography itself. Thanks MLA!

How can I get pages authenticated from the Wayback Machine? How can use the pages in court?

The Wayback Machine tool was not designed for legal use. We do have a legal request policy found at our legal page. Please read through the entire policy before contacting us with your questions. We do have a
standard affidavit
as well as a FAQ section for lawyers. We would prefer that before you contact us for such services, you see if the other side will stipulate instead. We do not have an in-house legal staff, so this service takes away from our normal duties. Once you have read through our policy, if you still have questions, please
contact us
for more information.

The Internet Archive

Societies have always placed importance on preserving their culture and heritage. But much early 20th-century media -- television and radio, for example -- was not saved.
The Library of Alexandria
-- an ancient center of learning containing a copy of every book in the world -- disappeared when it was burned to the ground.

What is the nonprofit status of the Internet Archive? Where does its funding come from?

The Internet Archive is a 501(c)(3) nonprofit organization. It receives in-kind and financial donations from a variety of sources, including, but not limited to: Alexa Internet, the Kahle/Austin Foundation, the Alfred P. Sloan Foundation, the William and Flora Hewlett Foundation, and
you.

Does the Archive issue grants?

No; although we promote the development of other Internet libraries through colloquia, and other means, the Archive is not a grant-making organization.

How do I get assistance with research? How about research about a particular book?

The Internet Archive focuses on preservation and providing access to digital cultural artifacts. For assistance with research or appraisal, you are bound to find the information you seek elsewhere on the internet. You may wish to inquire about reference services provided by your local public library. Your area's college library may also support specialized reference librarian services. We encourage your support of your local library, and the essential services your library's professional staff can provide in person. Local libraries are still an irreplaceable resource!

Audio

First, make sure you're logged on to archive.org with the same email address you used to upload the item.

The image you upload must be named
identifier.jpg (where
identifier
is your item's identifier name) and you must choose file format JPEG in the metadata editor.

To upload the image:

Go to your item's details page

Click the "Edit item" link in the lower left box

Upload the .jpg

After a few minutes, return to your item's details page. Click "Edit item" and find the .jpg file you just uploaded in the list of files near the bottom of this page. Select the file format JPEG from the drop down menu, and click the submit button.

Wait 5-20 minutes for your changes to show up. If you're still not seeing your new file, please try clearing your cache and viewing the page again, since you may still be looking at an old version of the page.

How can I get iTunes to create a new playlist when I stream MP3s?

As an iTunes user, you might have noticed that iTunes loads the Archive's streaming MP3s (M3U files) into your library, and subsequentially the files get shuffled and are out of order. We have come up with a solution to this problem.

There are a variety of problems that may be causing this. Here are a couple of the most common. If you have a Mac running OS X, the default unzip utility (Stuffit) does not deal well with those Archive ZIP files that are 'compressed on the fly'. You may see an empty directory - if so, then try downloading Zip Tools for Mac OS X
and using the drag and drop software within that to unzip your download. [Make sure you save your download to your desktop before trying things on it.] If you're having any trouble with downloads timing out or being incomplete, especially on Windows, then you may be able to use download managers such as GetRight. These will restart your download if it fails. However, some 'ZIP on the fly' downloads don't play well with download managers. If you find that to be the case, the safest thing to do is to download each track individually in a download manager.

How can I add a logo to the upper right corner of my collection?

First, make sure you're logged on to archive.org with the same email address you used when you created your collection. Note: Images should have a height of no more that 72 pixels.

Find logo file at bottom of page, choose "Collection Header" from the drop down list and click submit.

It might take a few minutes for the changes to appear.

How can I get my tracks to show up in the right order?

The most reliable way to have your tracks appear on the page in the correct order is to name the individual files with track numbers, like this:
01_nameoffirstsong.mp3
02_nameofsecondsong.mp3
03_nameofthirdsong.mp3

(If you have more than 9 files you need to start numbering with 01 - not 1 - otherwise the files will go in this order: 1, 10, 11, 12, 2, 3 etc.)

If you have already created an item and you would like to change the file names to rearrange them correctly, do the following:

Click the "Edit Item!" link

Rename your original files using track numbers

Delete all "derived" files, leaving only your original files and the .xml files

Click "Edit item" > "Item Manager" and then click the "derive" button

It will take a little while for the derive to finish running, but once it does you'll have all new files, in the correct order, in both the flash player and the page itself.

What kind of audio file should I submit?

The archive is all about free access to information, so you should submit file formats that are easily downloadable and/or streamable for other site patrons.

We prefer that you submit the highest quality file that you have available, and then we will attempt to create smaller file sizes and formats automatically with our deriver program. We recommend that you do not attempt to do any special encoding of your files - the more settings you mess around with, the less likely our deriver code will be able to process the file.

If you are submitting a Live Music Archive item, please only submit Flac or Shorten files. Even for non-LMA items, these are the best formats to use.

Whatever format you choose, please upload each file to your item individually (you can submit multiple files per item), in a non-compressed format. Uploading content in a .zip or .rar file makes your item unstreamable and significantly less accessible to others. If you upload .zip, .rar, non-audio formats (like .exe), or password-protected files, they may be removed by our moderators.

The table below describes what file formats we will attempt to derive depending on what type of file you submit.

This is automatically generated.
NOTE: inner whitespace is significant.
(*) Some derivatives only apply to certain collections.

Advanced techniques and help

To remove and/or prevent a particular derivative format

If for some reason, you prefer to not have your item create the derivative of one or more formats, you can upload to your item a special "rules" named "_rules.conf". That file should be a text file, with a single line of each format to disallow.

So, for example, to make (all videos in) a video item *not* create our "Animated GIF" and "h.264" formats, you would upload "_rules.conf" containing the following:

To prohibit *just* video and audio derived formats that are "lossy" (eg: mp3, ogg, mp4, ogv) you would upload "_rules.conf" containing the following:

To prohibit *all* derived formats, you would upload "_rules.conf" containing the following:

To make any previously created derivatives "disappear" after adding or updating a "_rules.conf" file to an item, use the "Item Manager" link to submit a "derive" task (which will remove the undesired derivatives). (Find the "Edit Item" link in the upper right of your item while you are logged in, click "change the information" link, click the "Item Manager" link near the top of that page, then hit the "derive" button).

The flash player is covering my files! How do I move it?

If an item has little or no description, sometimes the flash player doesn't have enough room in the top portion of the page and covers the files below. If you don't want to add a description (which would be nice, so that people know what they're listening to), you can add extra space in the description field using paragraph tags.

Live Music Archive

A recording I uploaded and marked 'no lossy formats' had them created (mp3, ogg, m3u, etc...) . How can I remove them?

If you come across this situation and you are the uploader, click [edit], select the derivation option you prefer, and then 'Update'. You should see the message "Format Options Updated Successfully". Within 10 minutes the system will create a "_rules.conf" file in the recording's folder. Then, the next time the system performs an automatic sweep looking for changes, it will notice the new rules file and remove the lossy files automatically. The sweep occurs approximately twice a day, so you should see the files removed within 12-24 hours.

If you are not the uploader, send us an email (etree at this domain) and an admin will remove them.

What is the Live Music Archive all about?

This audio archive is an online public library of live recordings available for royalty-free, no-cost public downloads. We only host material by trade-friendly
artists: those who like the idea of noncommercial distribution of some or all of their live material. Live recordings are a part of our culture and might be lost in 100 years if they're not archived. We think music matters and want to preserve it for future generations.

The LMA draws strength from the members of etree.org
and other online communities of music fans devoted to providing public access to high-quality digital recordings of tradable performances. Typically, recordings are made by the fans themselves. Recordings are preserved in "Lossless" archival compression formats such as Shorten or FLAC (MP3 is not Lossless) for highest quality preservation.

Patrons may download from the LMA with the understanding that the artists still hold their copyrights. All material is strictly noncommercial, both for access here and for any further distribution.

What are MD5 files?

MD5 files contain checksums, strings of characters used to uniquely represent a file. These checksums enable users to verify that music files downloaded correctly.

A recommended tool for creating these files is MD5summer. Please note that before uploading the MD5 created with this tool you should open the MD5 in a text editor and remove the top 3 lines so the first signature is now flush with the top of the file.

What are FLAC files and how can I listen to them?

FLAC stands for free lossless audio codec. It is an open source, lossless compression algorithm for digital music. It compresses music files to 50-60% of their original size, with no loss in quality. More FLAC information can be found on the FLAC sourceforge site
and in this etree FAQ.

If you upload FLAC filesets to the LMA, please follow the naming standards
to help the checking program here. Directories should be named with .flac16 or .flac24 suffix, not .flac. Otherwise, the program will report failures.

We'd like to make sure that a trade-friendly
band would not mind having their shows in the Archive for public download. The best way for us to find out is by getting permission from a band representative or by the band's having an explicit policy that covers this type of site. If there are no shows by the band, either we don't have enough of this information to go forward with archiving, they have declined participation, or we are ready to accept shows but no one has uploaded anything yet. (Also, see the band status FAQ).

Trade-unfriendly bands will not be found in the Archive, nor will otherwise trade-friendly bands who have declined to have material archived here.

Bands, see other relevant FAQs here
and here. Patrons, see more about how you can help
here.

There's no setlist for this show - OR - The setlist does not match up with the number of files. Should I submit an error report?

There has been an increasing number of shows uploaded to the Live Music collection without setlist information, or the setlist was not properly matched to the files. When you notice a recording like this, please email us (etree at this domain) only
if you have an updated setlist, or you are able to match the files up correctly.

We would prefer that you do not submit error reports letting us know that there is no setlist - tracking down setlists for every concert and matching them up to the recordings is a monumental task that has grown beyond the capabilities of the small group of Archive.org admins. We would like fans that are familiar with each artist's material to help us with this project - in your email, please give us specific instructions on what changes to make and we will do so.

How do I burn FLAC files to CD as audio tracks?

You will first need to convert the FLAC files to another format that your burning program is familiar with. Windows users can use the FLAC Frontend, to convert FLAC files to WAV files, which are suitable for burning programs. For Macintosh OS X users, Scott Brown has created a tool called
xACT.

How do I burn SHN files to CD as audio tracks?

You will first need to convert the SHN files to another format that your burning program is familiar with. The following programs will convert SHN files to WAV files, which can be burned to a CD. More resources are listed in this FAQ.

Linux or any other UNIX-based architecture: Download and install shorten.

How can I add a logo to the upper right corner of my collection?

First, make sure you're logged on to archive.org with the same email address you used when you created your collection. Note: Images should have a height of no more that 72 pixels. • Go to your collection's front page • Click the "Edit Item!" link next to your user name. • Click "Item Manager" near the top of the page. • Click the "checkout --edit items files (non XML)" button in the "Edit Operations" section of the form. • In Step 1 of 2, click the "Share" button. • Locate and select the image to be uploaded and click "Select". • In Step 2 of 2 click the "Update Item!" button • Return to collection front page and click "edit" link again • Find logo file at bottom of page, choose "Collection Header" from the drop down list and click submit. It might take a few minutes for the changes to appear.

What is the status of band X for the Archive?

5/2006, significant site changes in progress: Formerly, you could check on the status of a band relative to the Archive on the
Trade-Friendly Band Information
page, which is no longer updated. This FAQ question has been updated for the new-system presentation of info. We have 3 categories:

May be Archived-
Band sections have been activated by Archive admins. Shows can be hosted here to the extent permitted by the band. Click on the band name
and then through to their Policy Notes link to see what limits
they may have placed on taping, trading or archiving.

Pending-
When a patron sends us information about having contacted an additional trade-friendly band, the new band is considered to be "Pending". Admins will update notes
we keep on the band based on the information that people send to etree at archive dot org. (Sensitive parts of the info- such as email addresses used- will not
be posted in the public notes.)

Important: Under the new system, we cannot create a "collection page" for the band name unless and until we know that the band May Be Archived. Further, no shows may be uploaded for any band in advance of a band section's activation. Under the new system, there is no temporary "upload area" to store filesets for bands whose sections are not prepared yet. Please send shows for
bands on the active list
only.

Opted Out-
Some bands that may be otherwise trade-friendly may have explicitly said, "No, thanks" to our project. We respect their wishes. We still keep notes of their taping/trading policies for reference.

If your favorite band name is not in any of these 3 categories, there are several possible reasons: They may not be trade-friendly in the first place. No one may have contacted them yet. Someone who contacted them may not have informed us yet. The band may not have written us back yet. If a band did write to us, we may not have had a chance to activate a section yet, or we may not have received enough information back from them to setup their section. In some cases, we may not have received the email successfully, so that a resend may be necessary.

Bands, see other relevant FAQs here
and here. Patrons, see more about how you can help
here.

I'm an artist who would like to be included in the Archive, what do I need to do?

We'd love to have you! Just write to us at etree at archive dot org in English giving some kind of permission for us to archive your shows for public download and noncommercial, royalty-free circulation. It does not need to be a formally worded declaration, and can come from anyone you feel has the "say-so." We just need to be clear on how you feel about the project. We will put relevant quotes onto a new "collection" page (examples) for your performances, along with a link to your official website.

It is necessary for you to
email us
at etree at archive dot org in order to create a new section. We want to be sure that the go-ahead really is coming from you. Please do not attempt to create your own collection, or to upload any of the band's shows, in advance of receiving an emailed confirmation message from curators; such attempts may significantly complicate or delay the curators' setup process.

You can give as much or as little scope for archiving as you like. Some bands place limits
on what can be hosted, and we can accomodate those. Archive Curators, volunteer fans who have proven to be in line with the spirit of this archive, will attempt to screen contributions for OK'ed material only.

At the same time you give the go-ahead, feel free to pass along any notes or policy links on your general taping/trading stance as well. You don't need to have a formal written or posted policy before inclusion, but we'd like to know how you feel about the topic.

Besides fans' sending their copies of your shows, you can also prepare
and upload
your own live recordings to the Archive, if you like. In fact, if you'd like to limit your material to selected contributions from you only, please just let us know.

If you have any questions about the project, please ask us anytime at etree at archive dot org.

Can I upload concert videos?

At this time, video uploads are not being accepted, namely because most of the bands archived prohibit the video taping of their shows. Moreover, unlike audio, where we actually have a shot at archiving the vast majority of any given band's live concerts (in very high quality format), video is scarce and, unless made by the artist (in which case, it's typically for commercial purposes), is not of particularly good quality.

The progress of my upload says 'File metadata XML invalid. Waiting for user to correct.' How can I fix this?

This is typically caused by illegal symbols being used somewhere in the information that was put into one of the forms submitted with the show (either the import form or "File Options"). Double check that the only characters being used are those visible on a standard English-language 104 key keyboard. More information and a few examples are here.

If you have trouble finding the cause, please post to the forum for help. An admin will have to resubmit the recording for another try, so please send an email including a link to the recording to etree AT archive DOT org if you believe you have cleared the issue.

Feel free to email etree at archive dot org with any questions, and we'll do our best to post the answers here as soon as possible. Also, the message board
is a great resource; with so many kind, knowledgable folks out there, you can often get a speedy answer to your question.

I have a different source for a show that is already in the archive, should I upload it anyway?

Yes! In keeping with the nature of this Archive, it is appropriate for multiple sources of the same show to be available for download. When you upload the new source, be sure to name the source in the show's top level folder to avoid confusion. Some bands do place limits
on the types of sources allowed (such as soundboard recordings), so please check the policy
for any given band.

How can I help get bands into the Live Music Archive?

If you know of a trade-friendly
live-performing band that is a good candidate for the Archive, you can initiate contact. Some tips and letter templates can be found here. When you write, make it clear you are asking about the Live Music Archive at archive.org. Don't just ask about their general taping/trading stance. We want bands to know what's up.

Next, follow up with a message to etree at archive dot org. Mention when you tried to contact the band and what contact point you used. These are important in order to update our contact records. Admins will update the contact status in an announcement forum about Pending Bands
based on the message you send us.

If you receive a reply from the band, positive ornegative, send a complete copy of the email, complete with its sender's address/brief header info, to etree at archive dot org. It's a good idea to send a copy of what you asked them as well (if not quoted in the reply), since it will give context to the answer. We need to have full info in hand in order to set up the band appropriately in the Archive, and we may need to contact them for followup questions.

If you are hesitant to make contact yourself, you can mention the band to Archive admins (send email to etree at archive dot org) and they can try a contact as time permits. To help out, supply any contact or policy info you may already know about the band.

When I download concerts, I constantly get disconnected before the download completes. What can I do to fix this?

Most web browsers now support robust http downloading. For questions, see the support website for your browser.

What are the WAV MD5 files that are sometimes in filesets?

MD5 checksums files are not exclusive to SHN files. An MD5 checksum can be used to ensure the accuracy of any
data file (e.g. .doc, .mp3, .mpeg). Some seeders produce MD5 checksums for their WAV files, as well as for their SHN files. This is just an extra level of confirm to ensure exact copies of the original WAV files are being burned from the SHN files. Checking a WAV file with a MD5 cheksum is no different than checking a SHN file. If you use mkwACT, you can just right click on the wav MD5 and choose "verify."

I just uploaded a directory that contained WAV MD5 checksums, is that OK?

The WAV MD5 checksums are ignored by our robot and will not cause problems for your recording.

My failure email is indicating that the text file failed. What can I do?

Unlike FLAC or SHN, text files do not translate identically from 1 platform to another. Since the archive.org servers run Unix, text files created on other Operating Systems will fail their MD5check. We recommend uploaders remove any text files from their MD5's if they are having this problem.

Can bands place restrictions on material to be archived?

Yes. Each band can tailor the extent of their permission to the Archive. We quote the band's wishes in the Rights section of the band's Collection page.
Here are some examples
of special restrictions bands have requested. We point out different cases in a band's policy information using a shorthand "Limited Flag" tag.

We have a contribution system set up to accomodate individual bands' requirements. During the upload process, contributors are urged to double check the band's policy notes at different stages. Archive Curators, volunteer fans who have proven to be in line with the spirit of this archive, will attempt to screen contributions for OK'ed material only. In addition, access to a particular item can be removed if it becomes restricted later (for example, a date newly chosen for commercial release must be removed under some band's policies).

Bands, please contact us at etree at archive dot org anytime to let us know how we can work with you to make things happen.

I just uploaded a show and all the files fail the MD5 check, what's the deal?

Please be sure that if you are choosing any upload format, you are uploading the files in "binary" mode. If you try to upload .shn or .flac files in "ASCII" mode the files will fail the MD5 check. ASCII is the standard format for encoding plain text files (actually a subset of binary), while binary is used to encode almost all other types of files. More information on binary vs. ASCII can be found here.

If this does not solve the problem, be sure that all the file names in the MD5 file match the .shn file names. Be aware that the UNIX system the Internet Archive runs on is case-sensitive.

If you upload FLAC filesets to the LMA, please follow the naming standards
to help the checking program here. Directories should be named with .flac16 or .flac24 suffix, not .flac. Otherwise, the program will report failures.

Where have all the Dave Matthews Band concerts gone? Will they be back?

At the request of the band's management and as a result of the band's 2003 policy change, Dave Matthews Band concerts (as well as Dave Matthews solo concerts and Dave and Tim shows) have been removed from the Internet Archive. We're very sorry about this unfortunate turn of events but feel like it is important to honor the wishes of the band and its management.

Phish has decided not to participate in the Archive at this point in time

Similarly, Widespread Panic has opted out of the project for the time being. They were last contacted on 11/9/2004.

I used to use a download manager and now it stopped working. What's the deal?

Download managers increase your download speed by connecting to the server multiple times. Doing this does not significantly increase download speeds but dramatically hurts the performance of the server. If you wish to use queue to download from the HTTP servers, be sure you set your download program to only use one connection
at a time.

What's the deal with magic number errors?

If you get a magic number error when listening to or decoding a SHN file, the SHN file is most likely corrupt. First, make sure the SHN file passes MD5 verification; if it does not, redownload the file. If the file passes MD5 verification and you are still getting the magic number error, leave am error report via the show details page noting the magic number error and which track the error occurs on. Hopefully others who have download the show will confirm or deny the error. If the error occurs for all downloaders, the seeder will be contacted to provide a new, uncorrupted track. Please note that there is nothing the Internet Archive administrators can do about a magic number error, becuase the only solution to the error is re-encoding the SHN file from the original WAV file.

Do you provide an RSS feed of new updates to the LMA?

Indeed! The URL of the feed is http://www.archive.org/services/collection-rss.php?mediatype=etree&collection=etree You can plug this into a front end like AmphetaDesk (available at: http://www.amphetadesk.com)

What does the 'Transferred by' field mean?

This field indicates the person who did the original DAT/MD/Cassette to WAV conversion. Also, note that in the case of recordings made directly to laptops there is no transfer.

Why don't I get an email when my uploads fail MD5 checksums?

The system currently only sends emails when MD5 files are included. This means that, if you're uploading FLAC files, you still need to generate and include an MD5 file if you want to receive informational emails about the failures.

A recommended tool for creating these files is MD5summer. Please note that before uploading the MD5 created with this tool you should open the MD5 in a text editor and remove the top 3 lines so the first signature is now flush with the top of the file.

Can I log into an FTP server to download concerts?

Update (2009April): To allow us more flexibility on access, we are discontinuing FTP read access. HTTP read access (as in downloading through your web browser), remains more popular with users, and shall continue.

There are 2 XML files that get created during the import of any recording in the collection:

showfolder_meta.xml
showfolder_files.xml

The first file gets created when you submit the import form to the collection. If that file does not exist, you can create it by editing the details page and clicking Update.

The second file gets created by filling out File Options. Just click the link on the left side of the details page and fill out the form as accurately as you can.

If either of these files are missing, your Contribution may give you this message. Please note that once the files get created, it takes 5-10 minutes before the system notices them and moves on to the next stage.

Can I upload live recordings that were broadcast on XM Radio or Sirius Satellite Radio?

At this point in time, Archive.org cannot host recordings that were broadcast over either of these services. Subscribers have informed us that they were required to sign a "Terms of Use" document that forbids the recording/hosting/rebroadcasting of any material received from these services. Until we hear otherwise, these recordings cannot be hosted here.

The Grateful Dead is here, when will we see Jerry Garcia recordings?

The taping policy of the Grateful Dead does not extend to recordings of Jerry Garcia's other lineups. Jerry's solo work is controlled by his estate. Representatives have said No to the idea of hosting shows in the Live Music Archive.

Regarding removing the lossy files ... I edited my show, checked the box to remove them and clicked update. Now when I click update again, the box is still not checked. Why?

The upload instructions require a 'FLAC Fingerprint' file with my recording - how can I create this?

In Windows:

1. Open FLAC Frontend
2. Drag all of the FLAC files of your recording into Flac Frontend window. (you can also use the "add" button to do this)
3. Click the "Fingerprint" button.
4. Save the fingerprint file with a name like this: bandYYYY-MM-DD.ffp

I've got a great 'filler' for the recording I am about to upload to the collection - should I include it?

A 'filler' is music from a different performance in addition to the main recording, typically used to fill up extra space on a CD. Sometimes the filler is a different artist, other times it is the same artist, but a different show and date.

While this is convenient for burning full CD's, it is not appropriate to include fillers on recordings here in the collection since they get filed under the artist and date of main performance. Please only include the performance for the artist and date you are importing. Fillers should be filed under their own entries elsewhere in the collection.

Where can I find other recordings by [trade-friendly band] that aren't in the collection?

If the artist is OK with Internet trading, you may be able to find downloadable recordings through http://bt.etree.org. Also, check
http://db.etree.org
to find people who have copies of shows and who may be willing to trade. Etree.org has additional trading forums at http://forums.etree.org
Lastly, you can check out a band's own fan forums and mailing lists. Good luck!

In contrast, the Live Music Archive forum at the Internet Archive is
not
a good place to post about trades, or to ask for shows that are not yet archived here, whether or not the band presently has a section here. Moderators may delete these posts. More posting etiquette tips for that forum are here.

I tried downloading a show and I got a '403 Forbidden' page. Why?

As part of the new (as of May 2007) QA/QC checks that the archive conducts on shows that are uploaded, more refined checks are conducted on shows. For more detail, see this forum post: http://www.archive.org/iathreads/post-view.php?id=124098
What happens though, when a show either fails it's md5 check, it's internal flac checksum check, or is missing an info.txt file, every non .xml file in the show fileset (the flac files, the mp3's, etc) all become non-downloadable. If you try and click any of the music files, you will be taken to a webpage titled "403 Forbidden" that will say: "Forbidden You don't have permission to access "ARCHIVE.ORG_Server/show_location/file" (specific to your show file) on this server. **** What this means is that the uploader has a problem with their show files, and as a measure to 'stop the spread' of bad files, the system is preventing people from downloading until the uploader contacts the archive to fix the show. If you as a user find a show that has the above problem, please check back later and once the uploader has fixed the problem, the show will be downloadable as normal.

Next, be sure that you are logged in as an Internet Archive member. Have the fileset on your computer already, correctly prepared
and correctly named. Files
must
be in lossless format
(.flac or .shn), from lossless
parent source material; we will optionally create the extra "lossy derivative" copies (.mp3, .ogg) onsite. Prepare to create an item, following example tips
here
or here.

How do I make corrections to shows?

Sometimes people make typos or other mistakes on uploads, or leave gaps in info that can be filled in later. You can help supply good information for archived items. Here is the current best method to submit corrections:

If you uploaded the show, you can make the changes to the details page yourself. Make sure you are logged in as the user who uploaded the show and go to the details page of the show you are trying edit. Click on the "edit" link next to the band name at the top of the details page and you will be able to edit the show details including venue, location, source, setlist, etc. Be aware that editing these fields will only change the show details, not the files themselves.

5/2006 update: If you uploaded the item and would like to replace or add to files within your item, under the current system this can be done without reuploading the entire fileset. More description may follow; meanwhile there is a walkthrough as a
Word document with screenshots. Specifically to fix your items derived between 5/11-22/2006 that sound too fast in the onsite flash player (chipmunk problem), see this
PDF document with screenshots.

If you did not upload the show, please email the admins (etree at this domain), and state precisely what the problem with that particular show is. If the problem is a missing setlist, please see this FAQ). If there are one or more missing or broken files that you can provide, please re-upload and re-import the entire show under a new directory name, and then email us a link to the old, broken show, asking for that show to be removed.

What file formats are accepted for contributions to the Live Music Archive?

Currently, the Live Music Archive will only accept audio files in either of two lossless formats: FLAC (.flac) or Shorten (.shn). Please Note that MKW files (.mkw) are *NOT* an acceptable file format for your contributions because they lack cross-platform compatibility (Mac users are unable to play or decode MKW files)

In addition, please do not upload the lossy files (MP3 or OGG) next to your FLAC or SHN format files - the Archive creates
those files automatically, provided that the contributor agrees to having them available. This ensures that all the files here have uniform quality options selected.

Please follow etree.org's Seeding Guidelines
when preparing your contributions for addition to the collection. Pay particular attention to the Naming Standards
section. A well-named identifier helps patrons find your show in our large collection. A well-named set of files allows files to be listed in the proper order at the site, and allows patrons to listen to them in playlists and burn them to CD in the proper order, too.

I like adding concerts. Do you have a preference on the way I put in information?

First of all - thank you so much for contributing to the Archive. Yes, here are some guidelines that will help us maintain good records for each concert.

Do not include HTML in the source and lineage fields.

Do not repeat information in the notes fields (such as source information, or number of discs). Only include information in the notes fields that is not already in any other field.

If at all possible, keep absolutely nothing but song names in the setlist (even things like disc splits, set splits, etc. should not be in this field). If possible, putting all song names on one line, separated by commas is wonderful.

Do not fill in unknown field with questions marks or N/A - just leave them blank. The exception to this guideline is the venue, setlist and source fields (which are mandatory) - in the event that this information is not known, simply write "unknown".

The Grateful Dead is being separated from the Live Music Archive
into its own collection
(with its own forum) to avoid confusion about lossless availability. The metadata and reviews for shows and recordings, even those not available for regular download, will remain available for those who maintain direct links. No filesets have been deleted from the Archive; certain items are simply not public now. Prior to our completing the changes,
text files
are easily referenced at a separate database.

At this time, the Grateful Dead collection is not open to public uploads. The Grateful Dead Internet Archive Project
(GDIAP) will continue its direct management of this collection for the time being.

As far as we know, there has been no change to standard GD fan trading. It is common for bands to have policies
that differ between fan trading, versus archiving here.

What are the options for streaming a full recording?

Hi-Fi:
An MP3 playlist, readable by most players, that has the addresses of MP3 files encoded with a variable bit rate.

Lo-Fi:
An MP3 playlist, readable by most players, that has the addresses of MP3 files encoded with at a constant bit rate of 64 kilobits per second. These files are ideal for users with slower Internet connections.

What are the options for downloading a full recording?

Update 5/2006:
Please note that due to a major system transition, many items' ZIP files (for their "Lossless" links) have been deliberately disabled
for the time being. Engineers are still working on the best method for the new system.

Lossless:
A ZIP file containing Shorten files or Flac files. Unlike formats like MP3, lossless formats are true to the original - there is no degradation in quality.

Hi-Fi:
A ZIP file containing MP3 files encoded with a variable bit rate to deliver high quality at roughly 160kilobits per second.

Lo-Fi:
A ZIP file containing MP3 files encoded at a constant bit rate of 64 kilobits per second. These files are ideal for users with slower Internet connections.

Other Web Options:
All files are displayed as individual links on any item's details page. Web-based download managers can be set up to download all the files you want from the page, as a group. For Firefox, the extension
DownThemAll
is a popular option.

BitTorrent:
Some Items that are downloadable via HTTP are also downloadable via a BitTorrent client; these items show a 'Torrent' link next to the 'HTTP' download link. (To trigger creation of a BitTorrent file for an item in the LMA that does not yet have one, write a review for it, e.g. "Make me a Torrent!"). Note: only items downloadable via HTTP can be downloaded via BitTorrent.

Where can I see the rest of the 'Most Downloaded Items' in the Live Music Archive?

Please email the logo to the admin email address (etree at this domain). Images should have a height of no more than 100 pixels, and be no wider than 200 pixels.

How are download counts calculated?

Downloads are calculated per item page, per IP address, per day. If you stream a show today, that's one download. If you view the txt file tomorrow, that's another download. If you download every file from a show's page the next day, that counts as one more download. If you download the same file a thousand times the day after that, that still only counts as one more download.

Texts and Books

I'd like to upload a book. What format should it be in? How do you do your sponsored scanning for Contributing Libraries?

Probably the simplest way to contribute a text item currently is as a pdf. That way, the entire set of images can be submitted as a single file, and there are no special naming requirements, beyond ending the filename with ".pdf". If the pdf has no hidden text layer (i.e., isn't searchable), then after doing OCR, Archive.org creates a second pdf with a text layer.

Items can also be submitted as a stack of image files, one image per page. The files can be in JPEG2000, JPG, or TIFF format. We plan to provide a more flexible intake procedure, but at present, there are rather strict requirements for how the files in an image stack are to be named, and the stack needs to be packed into a single .zip or .tar file before submission.

When Archive.org scans a book for a Contributing Library, we use the custom-engineered "Scribe" workstation, but for many materials, adequate images can be made with off-the-shelf scanners or good-quality digital cameras. For best results, use the highest resolution your device is capable of. Most images we process were produced at a resolution of 300-600 ppi.

ePub is an open textual format (not images of pages). Many readers are becoming available. A free one is from Adobe.
Mobi is a proprietary textual format from Amazon supported on the Kindle.
DJVU is an open format for scanned documents with free readers for windows, mac os-x, linux.
It is compact, searchable, good looking, and open format.

Internet Archive's Internet Bookmobile is currently out of commission.

What is the directory structure for the texts?

Note re the instructions below:

"XXXX" stands for a 4-digit sequence number, starting with 0000.

What you're uploading is technically considered "processed" images, not "original" ones, even though they are in fact the originals, because archive.org processors wouldn't be doing any rotating or cropping.

The zip or tar has to be built from the parent directory, so that the directory name is included as part of the filename of each file stored in the zip/tar.

In order to store all the texts that the archive has, and will eventually acquire, the directory structure is:

IDENTIFIER/IDENTIFIER.extension (tif, djvu, pdf)

IDENTIFIER: Unique in Archive's collection, alphanumeric (URL safe), this is the original name adopted by the originating collection (alphanumeric characters and _-. Best if from 5 to 80 characters). One format is [title:8-16][vol:2][author:4][scanninglocation:0-4]

EXTENSIONS:

If the original files are tif files, then:

IDENTIFIER_orig.tif: All the orginal tiffs are stored in the form of multi page tiff. Demoware windows viewer Informatik Image Viewer. If it goes over 2GB, then it is stored as a tar of singlepage tifs the directory named IDENTIFIER_orig_tif/IDENTIFIER_orig_XXXX.tif resulting in a file called IDENTIFIER_orig_tif.tar

IDENTIFIER.tif: All the cleaned up tifs (usually cropped, despeckled, deskewed) are stored in the form of multi page tiffs. If it goes over 2GB, then it is stored as a tar of a directory named ./IDENTIFIER_tif/IDENTIFIER_XXXX.tif resulting in a file called IDENTIFIER_tif.tar

If the original files are JPEG JP2 or CR2 files, then:

All the original jpg files are used to make a zip file named IDENTIFIER_orig_jpg.zip where the names of the pages in the zipped directory are IDENTIFIER_orig_jpg/IDENTIFIER_orig_XXXX.jpg. If the resulting file is greater than 2GB (thus breaking the zip format until zip64 is common), then the file will be in tar format named IDENTIFIER_orig_jpg.tar . If the originals are jp2 or cr2 files, then substitute these extentions above.

Similarly all the processed jpg files (cropped and deskewed) are used to make a zip file named IDENTIFIER_jpg.zip where the names of the pages in the zipped directory are IDENTIFIER_jpg/IDENTIFIER_XXXX.jpg. If the resulting file is greater than 2GB (thus breaking the zip format until zip64 is common), then the file will be in tar format named IDENTIFIER_jpg.tar

In the case where there is a small jpg version of the files for on-screen access then a similar naming convention is used from the _orig.jpg version above, but with _200KB resulting in a file named IDENTIFIER_200KB_jpg.zip where the names of the pages in the zipped directory are IDENTIFIER_200KB_jpg/IDENTIFIER_200KB_XXXX.jpg. An equivalent version can be done with other sizes and different formats such as jp2.

IDENTIFIER.djvu: A nifty open scanned book format created by AT&T Labs and enhanced by LizardTech.com enabling compression and ease of reprinting. This file will also be ocr'd to make the text searchable.( /djvu/bin/documenttodjvu --filelist.txt temp.djvu, /djvu/bin --ocr aatttt.djvu)

IDENTIFIER_djvu.xml this is an xml version of the OCR output which has the word positions (as a bounding box). this is used for building the djvu file, and is used for searching the flip books, and maybe constructing a searchable pdf in the future.

IDENTIFIER.pdf: Adobe acrobat format that is derived from the .tif file if present.

IDENTIFIER.txt.tar.gz or .art.tar.gz: If there are OCR'ed text files associated with each page, these are tarred and gzipped in txt format or art which is sakhr format.

IDENTIFIER_cover.doc or .sxw:
cover of the book, some in legal and some letter. doc is Microsoft Word, and sxw is OpenOffice.

IDENTIFIER_xxxx_bookplate.jp2 or .jpg: is the file that has a bookplate that acknowledges those behind creating the digital version. xxxx is the page that it will replace in the access formats.

IDENTIFIER_meta.xml: This has the catalog data (title, author, publisher, copyright information) and information about the book found while scanning (size, who scanned it) stored in a dublincore-like XML format.

IDENTIFIER_meta.mrc: This will be the MARC (Machine Readable Cataloging) records for the book which provides the mechanism by which computers exchange, use and interpret bibliographic information and its data elements make up the foundation of most library catalogs used today.

IDENTIFIER_marc.xml: marcxml format of marc record

IDENTIFIER_metasource.xml: where the metadata information came from (metadata about the metadata :) ).

LEGACY FORMATS: This could be OTIFF | PTIFF | TXT.

OTIFF: These are the original tiff images of the scans of the books. (to create multipage tifs we used a unix util: tiffcp OTIFF/*.tif aaattt_orig.tif)

PTIFF: These are processed images (cropped,desqewed,depeckled) from the originaltiffs.

TXT: These are the text files that have been created by doing Optical Character Recoginiton (OCR) on the tiff images.

* We plan to eventually remove OTIFF|PTIFF|TXT directories.

What is OpenLibrary? How can I make my book available via OpenLibrary.org?

The Open Library is a project of the Internet Archive (archive.org), a non-profit organization in San Francisco, guided by the goal of universal access to human knowledge. Our small team is working to create a web page for every book ever published, at openlibrary.org.

Some facts about Open Library you might like to know:

You are free to edit/correct any errors or omissions you see on openlibrary.org - it's an open, editable wiki. (Just look for the "EDIT" button.)

We serve a catalog some 23 million books, but not the books themselves.

We don't buy or sell books

We have no way of putting you in touch with authors or publishers

Our team isn't able to help you do research on titles you find in Open Library

Many
authors write in to ask how they can make their book available as a free download via OpenLibrary.org. Here's one option:

Since OpenLibrary.org is a user-editable project, you can sign in to OpenLibrary.org to create a page for your book. You can upload the book to Archive.org (see information above), and link to the copy you upload to Archive.org.

You have the option of choosing a particular Creative Commons license for your work, or making a custom statement on what specifically people can or can't do with your item. Remember that if you wish people to contact you regarding use permissions, you'll need to provide contact information, such as a mailing address or website. Some uploaders choose to include this information in the description field.

How do you remove line breaks from the Gutenberg texts?

In Word use find and replace 3 times:

Step 1. Find two paragraph markers - ^p^p

Replace with a neutral character ~ or # or @

Step 2. Find one para markers - ^p

Replace with a single space

(This might take about 10-15 minutes on large files)

Step 3. Put 2 para markers back in - find ~

Replace ^p^p

What is the best way to link to a book?

Every book in the Archive has an identifier. For example, RomeoAndJuliet. To link to the book, you should use the following URL:

http://www.archive.org/download/RomeoAndJuliet

Can I volunteer for the book project?

Volunteers are welcome to come to our San Francisco location! We need your help to scan books! Please contact info at archive.org for more information.

I see some books from a series, but not all. How can I access the rest?

Many contributing libraries work with the Internet Archive to scan and provide online access to books.

To ask about whether there are plans to include additional volumes, or other particular books, you can contact the Contributing Library.

For all items at archive.org, the "identifier" is a unique sequence of letters (with numbers also permitted) that is the basic unit of identification of an item. It travels with the digital object, and is involved in all ways of accessing or otherwise referring to an item.

You see the identifier at the end of an archive.org URL (web address).

For this URL: http://www.archive.org/details/lifeworksofabrah112linc the identifier is "lifeworksofabrah112linc".

For sponsored scanned books, the Internet Archive uses a custom algorithm to generate each book identifier.

Example: hereismytitle00auth

Using this algorithm, up to 16 characters are pulled from the 245 field in the MARC record (MARC
is a library catalog record format), and these make up the first part of the identifier.

Then, whatever volume information the loader indicates shows up immediately after that (for monographs this will usually read 00). And then the first 4 letters of the creator are pulled from the MARC 100 field.

The algorithm also has rules that pull out any articles or punctuation to decrease the chances of duplicating an identifier.

If a duplicate identifier is generated, the person loading the book record at the beginning of the digitization process is notified, and manually edits it to make it unique.

Some Mac users have written to us suggesting MPlayer (OS X), BBDEMUX, and MPEG2DECX -- free on
www.versiontracker.com.

For more details, troubleshooting, and how to play movies on other operating systems, see this
how-to page.

Why do I get errors when I try to play a movie?

The best all-around, free player is
VLC Media Player
- it handles most of the movie files you will find on this site. If you're seeing errors when you try to play movies, please try downloading VLC and using that instead. This clears up many people's problems.

Here are some other possible problems:

There is heavy traffic to our site. If you experience a delay, please try again later or at a different time of day.

You're behind a firewall and the firewall software is attempting to modify incoming bits. Contact your network or firewall administrator.

Your Internet connection went down or timed out. Check with your ISP or network administrator to see if there's a special policy about keeping a connection live.

If your browser seems to hang after a "100% downloaded" message, check to see that you have sufficient hard-disk and TMP disk space. Rebooting the system sometimes helps.

You are trying to play an MPEG-2 file on a platform other than Windows or Linux. At present, you need VLC (
http://www.videolan.org
) or the for-pay quicktime6
add-on to play MPEG-2 files on the Macintosh. Please contact us at info at archive dot org if you have information about other players that work on platforms other than Windows.

2. Your player tried to stream the movie, and it isn't streamable. Download the movie first, and then play it. (Right-click > Save As)

3. Some conflict exists between your computer's configuration and the player you're using. Unfortunately, because PCs can be set up in so many different ways and because different standards exist for playing video, finding a player that will work is a hit-and-miss process. Try Rod Hewitt's
evaluations
of a number of players.

You can Re-encode Mpeg2 movies to quicktime for FinalCut Pro using Cleaner5.0.2 using the following settings. There is no de-interlacing, so you don't lose anything. The files increase in size 10 fold, so make sure you have enough HD space. This procedure gives you quicktime movies suitable for use with final cut.

-- NEW -- One of the simplest ways to transcode movies from MPEG-2 to DV format for editing is to use the freeware utility MPEG Streamclip (Mac OS X and Windows) available at squared5.com. It offers many settings and maintains video/audio sync.

Sometimes when I play a movie, the video is choppy or very pixelated. Why is that?

Try downloading the movie to your computer and watching it locally. Sometimes choppiness occurs when we can't stream it to you quickly enough (because your connection is slow or our servers are overloaded).

If you're watching an MPEG-4 that we derived from an original MPEG-2, we first reduce its size to 320 x 240 - a quarter of the resolution of NTSC video. We then translate it at 350 kbps, which is really borderline for that resolution. You see errors occasionally because there simply isn't enough bandwidth available, so the MPEG-4 encoder either drops frames - resulting in jerky or choppy motion - or drops macro blocks - resulting in blurred or pixelated video. That is the price we pay for the small file size - 80 MB for a 1/2-hour clip is really very small in the digital video world. If this is the case, download the original MPEG-2 to solve the problem.

Who owns the rights to these movies?

This will vary from movie to movie.

Many of the movies and collections are licensed with Creative Commons Licenses. Uploaders may designate whether or not an item has a CC License. If they do so, the Creative Commons logo will appear on the left hand side of the movie's detail page. Click on this logo to see details about the specific type of license that the uploader has assigned to the movie. Archive.org cannot guarantee the accuracy of uploader-provided information.

Some films may have the contact information listed for the filmmaker. If the information is provided, feel free to contact the filmmaker or organization the film comes from.

There are many sites that allow users to upload videos, but most of them only display very low quality video and/or do not let you download the videos.

As far as we know, this is the only site that presents high-quality downloadable movie data files with such liberal use restrictions. See the
Links page
at Prelinger Archives for a number of sites that may be useful to researchers or those seeking specific films or footage.

What are those animations associated with each movie and how did you make them?

The animations on the details pages and on the browse pages are animated GIF files. In most cases, still shots from each minute of the program were grabbed and saved as JPG files (these are the thumbnails which you can reach by clicking on the "View thumbnails" links). Then a tool called ImageMagick was used to create the animated GIF files from the JPGs.

We try to create an animated gif for every movie when it is uploaded (it may take a while to appear), but there are some file formats and/or encoding settings that make this difficult. If an animated gif hasn't appeared for your item by the day after you uploaded it, we probably couldn't make one for your item.

Can I stream the movies?

There are several programs you can use to stream movies in the Archive. Because we allow users to upload video files in any format, the same player will not always work for every single file, so it's a good idea to have a couple of programs available that you can try. Also, some files simply can't be streamed. Usually, this happens when the program that created the video file uses a codec that our software doesn't understand. So if you click on a stream link and get an "unsupported media" sort of error, use the download links instead.

Here are some free players that might come in handy:

Quicktime
If you have Quicktime installed, many mp4 streaming movies will play right in your browser window just by clicking a stream (or download) link. Make sure you have the latest version so that you can play the widest array of files.

VLC Media Player
Open your VLC Media Player and go to File > Open Network Stream. Click the File tab and enter the download
link of the file you want to watch. Yes, this seems backward, but it works!

So, if you were trying to stream the movie Duck and Cover found at http://www.archive.org/details/DuckandC1951 you would:
Use this URL: http://www.archive.org/download/DuckandC1951/DuckandC1951_256kb.mp4NOT
this URL: http://www.archive.org/stream/DuckandC1951/DuckandC1951_256kb.mp4

VLC will stream mp4, avi, mpg and other file formats, so it is quite useful for viewing the majority of the files in the archive.

We support two bitrates: 32Kbps-192Kbps for modem and ISDN users plus 256Kbps-450Kbps for DSL and cable-modem users.

Encoding Parameters

We attempt DVD, VCD, and MP4 streaming for broadband. We want these parameters to easily work with low-end video editors.

MPEG-2, DVD -- 720x480 or 702x480 interlaced. With a system header on each pack to be compatible with DVD. (Prelinger movies are 1/2 D1 352x480 29.97 fps which causes some players to make them look skinny)

An editable file is a file which can be downloaded and used in an editing program. The MPEG-4 are the highest bitrate versions we could do with the linux mpeg-2 to mpeg-4 conversion tools we use. These files can be read directly into FinalCut-Pro from Apple, and can be converted to mov using Quicktime-pro and read directly into iMovie from Apple.

The second command copies just the audio out of input.mpeg and produces output.audio:mplayer input.mpeg -aid 128 -dumpaudio -dumpfile output.audio

The third command combines the video and audio back together again in a format ready for dvdauthor:mplex -f 8 -V -o complete.vob output.video output.audio

This step creates the dvd structure. Create a new file with any text editor with the following:
<dvdauthor dest="DVD_folder">
<vmgm />
<titleset>
<titles>
<pgc>
<vob file="complete.vob" chapters="0,15:00,30:00,45:00,60:00"/>
</pgc>
</titles>
</titleset>
</dvdauthor>
The chapters line lists the points to include chapter marks on the DVD for jump navigation.

Now let dvdauthor create our dvd:dvdauthor -x dvdauthor.xml

Done! You should now have a folder called "DVD_folder" with your movie. You can create an ISO or BIN image with mkisofs:mkisofs -dvd-video -V "Movie Title" -o movie.iso DVD_folder/

You can play movie.iso in most any video player or burn it to a DVD:growisofs -speed=16 -dvd-compat -Z /dev/dvd=movie.iso

If you just want to burn the film to a DVD you do not have to create the movie.iso image file:growisofs -speed=16 -dvd-video -dvd-compat -V "Movie Title" -Z /dev/dvd DVD_folder/

Can I upload this movie?

You may upload movies that you own the copyright to, or that are in the public domain.

We are not copyright lawyers, and copyright is a tricky business, so you may want to consult a copyright researcher to clear material before you use it. You may also want to check this
list of movies
that one of our volunteers has already researched.

Here is some general information on the subject that may help you decide if your movie is okay to upload. The information below applies to films produced in the United States only.

1)
Is there a copyright notice visible in the film?
It is usually visible with the title or at the end of the film.

If the work was made in 1923 or earlier, it is probably public domain and can be uploaded.
NOTE!
Restored versions of the film or new soundtracks for silent films can have more recent copyrights that are still valid - usually a copyright notice for a new soundtrack or restoration will appear in the film.

For works made from 1923 to 1949, post a question to the movie forum on this site before you upload. The copyright could have been renewed and there isn't a way online to check a film's copyright status.

For works made from 1950 to 1963, you can check the title at the Library of Congress Copyright Database for copyright renewals:
http://www.copyright.gov/records/cohm.html
. This will list copyright renewals for most films.

If the copyright notice is 1964 or later, the copyright is probably still valid and the film should not be uploaded unless you are the copyright holder.

2)
Is the copyright notice in the correct format?
It needs to state three things - the word 'copyright' or the copyright symbol or '(c)', the year and who owns the copyright? If it is missing one of those elements or if there is no notice, it could be public domain. If you aren't sure, please post a question to the movie forum on this site.

3)
Is the film foreign (not from the U.S.)?
Foreign titles might not have a copyright notice, but still may be copyrighted in their country of origin. Traditionally the U.S. wouldn't recognize the copyright of a foreign film unless it was registered in the U.S. That has recently changed with the GATT treaty. Many foreign works had their copyrights restored. Please post a question to the movie forum on this site about these films before you upload.

What kind of movie file should I submit?

The archive is all about free access to information, so you should submit file formats that are easily downloadable and/or streamable for other site patrons.

We prefer that you submit the highest quality format that you have available, and then we will attempt to create smaller file sizes and formats automatically with our deriver program. MPEG2 files are the easiest file type for us to deal with. We recommend that you do not attempt to do any special encoding of your files - the more settings you mess around with, the less likely our deriver code will be able to process the file.

Whatever format you choose, please upload each file to your item individually, in a non-compressed format. Uploading content in a .zip or .rar file makes your item unstreamable and significantly less accessible to others. If you upload .zip, .rar, non-video formats (like .exe), or password-protected files, they may be removed by our moderators.

The table below describes what file formats we will attempt to derive depending on what type of file you submit.

This is automatically generated.
NOTE: inner whitespace is significant.
(*) Some derivatives only apply to certain collections.

Advanced techniques and help

To remove and/or prevent a particular derivative format

If for some reason, you prefer to not have your item create the derivative of one or more formats, you can upload to your item a special "rules" named "_rules.conf". That file should be a text file, with a single line of each format to disallow.

So, for example, to make (all videos in) a video item *not* create our "Animated GIF" and "h.264" formats, you would upload "_rules.conf" containing the following:

To prohibit *just* video and audio derived formats that are "lossy" (eg: mp3, ogg, mp4, ogv) you would upload "_rules.conf" containing the following:

To prohibit *all* derived formats, you would upload "_rules.conf" containing the following:

To make any previously created derivatives "disappear" after adding or updating a "_rules.conf" file to an item, use the "Item Manager" link to submit a "derive" task (which will remove the undesired derivatives). (Find the "Edit Item" link in the upper right of your item while you are logged in, click "change the information" link, click the "Item Manager" link near the top of that page, then hit the "derive" button).

How can I embed a flash player with my movie on my web page?

It's really easy to embed our flash player with your movie into your web site. To do so, go to the item page for the movie you want to embed. Then click the flash player as if to watch the movie. When you do, you'll see a small question mark beneath the player. Click on this and you'll get the instructions and code you need to embed the movie into your web page.

First, make sure you're logged on to archive.org with the same email address you used when you created your collection. Note: Images should have a height of no more that 72 pixels. • Go to your collection's front page • Click the "Edit Item!" link next to your user name. • Click "Item Manager" near the top of the page. • Click the "checkout --edit items files (non XML)" button in the "Edit Operations" section of the form. • In Step 1 of 2, click the "Share" button. • Locate and select the image to be uploaded and click "Select". • In Step 2 of 2 click the "Update Item!" button • Return to collection front page and click "edit" link again • Find logo file at bottom of page, choose "Collection Header" from the drop down list and click submit. It might take a few minutes for the changes to appear.

Questions

Downloading Content

Can I download files via FTP?

Update (2009April): To allow us more flexibility on access, we are discontinuing FTP read access. HTTP read access (as in downloading through your web browser), remains more popular with users, and shall continue.

Update (2009April):
Please note that to allow more flexibility on access, we are discontinuing FTP read access. HTTP read access (as in downloading through your web browser), remains more popular with users, and shall continue.

To download the files on a PC, right click the link to the file, and select "Save Target As" or "Save Link As" (or something similar depending on which browser you're using).

On the Macintosh, hold the button down while the mouse is over the link, and when the menu comes up, select "Save Target As".

Update (2012July): some Internet Archive items may be downloaded via the BitTorrent protocol using the link
Torrent
on the item's webpage. Download via BitTorrent requires an up-to-date BitTorrent client, see the FAQ on Archive BitTorrents
for more information.

Virtual Library Cards (AKA Accounts)

If I remove my account, will my items also be removed from the Archive?

No, your items will stay on archive.org once you delete your account. If you would like your items removed, please contact us at info AT archive DOT org.

I forgot my password, what can I do?

As long as you remember the email address which you originally used when signing up for your virtual library card, you can use this form
to have your password emailed to you. Bear in mind that your password will be sent in clear text, which means that anyone who views the email (or anyone with sophisticated "packet sniffing" software) can obtain your password. For this reason you should return to the Internet Archive website once you have your old password and change it to something new.

When I attempt to log in using my username and password, I am told that the username or password is invalid. What could be wrong?

There are several things to keep in mind when you encounter this error.

Your username is your email address, not your screen name. Make sure you enter the same email address that you supplied when signing up for your virtual library card.

Your password is case-sensitive. Check to see if the CAPS-LOCK key is engaged (typically a light would be illuminated on your keyboard).

You might have forgotten your password. If you think this is the case, you can have your password emailed to you
here

However, be aware that if you change the email address for your account, you will no longer be able to "edit" files posted from your old email address. If you would like to have your items' ownership transferred to a new email address, send an email to info AT archive DOT org from your OLD email address (the one you want to get rid of - that's how we know you own the items) and tell us which address you'd like to change it to.

An Open ID is not required to obtain a library card (account) for Archive.org

My account is locked. What can I do?

It is likely that your account was locked because you uploaded multiple items that seemed to have rights issues or the content you uploaded was inappropriate for the Archive. If you do have rights to the content you uploaded and you believe it is appropriate for Internet Archive, please contact us with your thoughts at info AT archive DOT org.

Questions

FreeCache

Why not Squid or mod_proxy?

Both Squid and mod_proxy are great for reducing the load on web servers, and we encourage everybody to use them. The disadvantage of these caching proxies are that they only work "vertically", i.e., they reduce the bandwidth downstream from the originating web site to the users' browsers. That web site still gets 1 download per (non-cascading) proxy. The FreeCache system works more "horizontally", i.e., FreeCaches fill themselves up from neighboring FreeCaches if at all possible. Hence, the load on the originating web site is much lower. FreeCache and caching proxies are complementary technologies. Both can be used to reduce the impact on web sites.

Why FreeCache?

FreeCache is a demand-driven, distributed caching system. Cooperating caches exchange files without burdening the original site too much.

Why not BitTorrent?

BitTorrent is good and similar to FreeCache in that it balances download "horizontally". BitTorrent uses other BitTorrent clients for this balancing; these clients often become un-available after a particular file is not popular anymore. The FreeCache system utilizes permanent FreeCaches that don't go away (although particular files get flushed out after a while). Unlike BitTorrent, the FreeCache system is transparent to the end-user. No new client or server software is required, and the files do not need to be converted. To offer a file via the FreeCache system, all you need to do is prefix the URL with http://freecache.org/

What files are being served by FreeCache?

FreeCache can only serve files that are on a web site. If the link to a file on that web site goes away, so will the file in the FreeCaches. Also, there is a minimum size requirement. We don't bother with files smaller than 5MB, as the saved bandwidth does not outweigh the protocol overhead in those cases.

What's a good download manager?

We like wget, because you can tell it to play nice and go slow. It's highly configurable and very powerful. Wget runs on all Unix platforms (incl. Mac OS X), and it comes standard with
Cygwin
on Windows. If you prefer something graphical, Mozilla's built-in download manager works fine.

DocuComp

What is DocuComp?

DocuComp is a sophisticated technology that compares inserted, deleted, replaced and moved text and content in Web pages. It's patented algorithm has been specially designed and licensed for use in the Wayback Machine.

What do I need I to know to use DocuComp in the WayBack Machine?

You only need to know the basic functions of the Wayback Machine. Begin by typing an URL into the Wayback Machine and hit the 'Take Me Back' button. Once you've found your choices on the results page, click the 'Compare Archive Pages' button in the upper right hand corner of the page. The reloaded page will have a series of check-boxes before each page date. Check any two dates and select the 'Compare two dates' button in the upper left-hand corner of the screen. The system is designed to automatically generate results for any URL's indexed by the Wayback Machine.

What Archive Pages are comparable?

You can compare any two pages from the Archive's library dating from 1996 to the present (approximately 55 billion pages).

Why should I compare results of past Web pages?

Access to the Archive's Collections is provided at no cost to you and is granted for scholarship and research purposes only. The DocuComp feature is intended to provide interesting insight into how content on pages in every field-- from the government to entertainment to business sites-- changes over time.

Where can I find out more about DocuComp?

Please visit the ww.docucomp.com
site. DocuComp is a widely-used technology that is licensed by it's parent company, Advanced Software, into many of the software products and content management systems available today. Formerly a standalone application for Advanced Software, the company now focuses exclusively on licensing the DocuComp technology and patent to software vendors.

How are images compared?

When compared pages contain different images, only the new (or latest) set of images is shown. Images that were either changed or removed are not displayed in the comparison results.

Some images are missing in my comparison?

In certain cases, images within the Web pages are not available. Not all images are archived nor are retrievable from the original site. If they no longer exist on the original site then the images will not be available and not displayed within the archived pages.

Certain links or actions are not working in the comparison results?

Links to other pages may not be live if those pages (or links) no longer exist and are not in the archive library. Also, javascript enabled links and actions are disabled in the comparison results to prevent errant scripts from being run.

How can I report problems?

After comparing two pages, the upper frame on the results page includes a hyperlink to report results which return any page faults. By clicking this hyperlink, an automatic error report is generated to both the Internet Archive webmaster and DocuComp's technical team. If you wish, there is an additional help screen to describe the issue. Please keep in mind that with over two billion pages to index and compare, not all being created alike; some pages will differ greatly and not have a common frame of reference to effectively compare.

Can I copy and use my results?

The results of any comparison done on the Internet Archive site are governed by the terms of use listed at: http://www.archive.org/about/terms.php. Additionally, any use of the DocuComp trademark or logo without express written permission by Advanced Software, Inc and any of it's affiliates is prohibited by law.

Guidelines for Press, Magazines and General Media

DocuComp is a registered trademark of Advanced Software, Inc. Please contact the company at (866) 329-7480 or info@docucomp.com
for background information on the company's history, technology data, or to schedule executive interviews.

Prelinger Movies

How did you digitize the films?

The Prelinger Archives
films are held in original film form (35mm, 16mm, 8mm, Super 8mm, and various obsolete formats like 28mm and 9.5mm). Films were first transferred to Betacam SP videotape, a widely used analog broadcast video standard, on telecine machines manufactured by Rank Cintel or Bosch. The film-to-tape transfer process is not a real-time process: It requires inspection of the film, repair of any physical damage, and supervision by a skilled operator who manipulates color, contrast, speed, and video controls.

The videotape masters created in the film-to-tape transfer suite were digitized in 2001-2003 at Prelinger Archives in New York City using an encoding workstation built by
Rod Hewitt. The workstation is a 550 MHz PC with a
FutureTel
NS320 MPEG encoder card. Custom software, also written by Rod Hewitt, drove the Betacam SP playback deck and managed the encoding process. The files were uploaded to hard disk through the courtesy of Flycode, Inc.

More recently, Prelinger films have been digitized and uploaded by Skip Elsheimer at
AV Geeks.

The files were encoded at constant bitrates ranging from 2.75 Mbps to 3.5 Mbps. Most were encoded at 480 x 480 pixels (2/3 D1) or 368 x 480 (roughly 1/2 D1). The encoder drops horizontal pixels during the digitizing process, which during decoding are interpolated by the decoder to produce a 720 x 480 picture. (Rod Hewitt's site
Coolstf
shows examples of an image before
and after
this process.) Picture quality is equal to or better than most direct broadcast satellite television. Audio was encoded at MPEG-1 Level 2, generally at 112 kbps. Both the MPEG-2 and MPEG-4 movies have mono audio tracks.

To convert the MPEG-2 video to MPEG-4, we used a program called FlasK MPEG. This is an MPEG-1/2 to AVI conversion tool that reads the source MPEG-2 and outputs an AVI file containing the video in MPEG-4 format and audio in uncompressed PCM format. We then use a program called Virtual Dub that recompresses the audio using the MPEG-1 Level 3 (MP3) format. This process is automated by the software that runs the system.

An article on re-coding Prelinger Archive films to SVCD so you can watch them on your DVD player.

Do I need to inform the Internet Archive and/or Prelinger Archives when I reuse these movies?

No. However, we would very much like to know how you have used this material, and we'd be thrilled to see what you've made with it. This may well help us improve this site. Please consider sending us a copy of your production (postal mail only), and let us know whether we can call attention to it on the site. Our address is:

Rick PrelingerPO Box 590622San Francisco, CA 94159United States

How can I get access to these movies on videotape or film?

Access to the movies stored on this site in videotape or film form is available to commercial users through Archive Films, representing Prelinger Archives for stock footage sales. Please contact Archive Films directly:

Please visit us at
www.prelinger.com/prelarch.html
for more information on access to these and similar films. Prelinger Archives regrets that it cannot generally provide access to movies stored on this Web site in other ways than through the site itself. We recognize that circumstances may arise when such access should be granted, and we welcome email requests. Please address them to Rick Prelinger.

The Internet Archive does not provide access to these films other than through this site.

What parameters were used when making the Real Media files on the website?

There are no restrictions. You are warmly encouraged to download, use and reproduce these films in whole or in part, in any medium or market throughout the world. You are also warmly encouraged to share, exchange, redistribute, transfer and copy these films, and especially encouraged to do so for free.

Any derivative works that you produce using these films are yours to perform, publish, reproduce, sell, or distribute in any way you wish without any limitations.

Descriptions, synopses, shotlists and other metadata provided by Prelinger Archives to this site are copyrighted jointly by Prelinger Archives and Getty Images. They may be quoted, excerpted or reproduced for educational, scholarly, nonprofit or archival purposes, but may not be reproduced for commercial purposes of any kind without permission.

If you require a written license agreement or need access to stock footage in a physical format (such as videotape or a higher-quality digital file), please contact Getty Images. The Internet Archive does not furnish written license agreements, nor does it comment on the rights status of a given film above and beyond the Creative Commons license.

We would appreciate attribution or credit whenever possible, but do not require it.

Largely because of copyright law. While a high percentage of ephemeral films were never originally copyrighted or (if initially copyrighted) never had their copyrights properly renewed, copyright laws still protect most moving image works produced in the United States from 1964 to the present. Since the Prelinger collection on this site exists to supply material to users without most rights restrictions, every title has been checked for copyright status. Those titles that either are copyrighted or whose status is in question have not been made available. For information on recent changes in copyright law, see the circular Duration of Copyright
(in PDF format) published by the Library of Congress

Questions

Search Tips

Every collection within Moving Images has a "Most Downloaded" list in the right-hand column of the page. However, if you'd like to see a complete list of all of our most downloaded movies, click here.

Can I see a list of the most downloaded audio files?

Every collection within Audio has a "Most Downloaded" list in the right-hand column of the page. However, if you'd like to see a complete list of all of our most downloaded audio items, click one of the links below:

Here's how to break it down. See the license types at
creative commons. When you want to find all of the items assigned a certain license by an uploading party, you'll plug their abbreviation for it into this search query:

/metadata/licenseurl:http*abbreviation/*

So if you're looking for Attribution Non-commercial No Derivatives (by-nc-nd), you'd put this in the search box:
/metadata/licenseurl:http*by-nc-nd/*
And you'd get about 33,000 items back.

If you want to use this in combination with other queries, like "I want by-nc-nd items about dogs" you'd do this:
/metadata/licenseurl:http*by-nc-nd/*
AND dog
And you'd get 195 items. The AND tells the search engine all the items returned should have that license AND they should contain the word dog. AND has to be in all caps.

Uploading Content

How can I add my music, movies, or text?

You may contribute content to the Internet Archive if it's in the public domain or if you own the rights to it. If you own the rights, we recommend that you choose a
Creative Commons license
for it so that others will know how they may (or may not) use it. You can choose a type of Creative Commons license during your upload process.

Please note that if you wish to be contacted with inquiries regarding your item, you'll need to supply public contact information. Some chose to provide a web address, mailing address, or other means of contact in the description text for the item.

This exciting new uploading feature is a beta release. This means that it's currently under development, and we've made this early release because it improves the upload process and we can't wait for you to start working with it!

To use the new beta uploader:

First click the "Upload" button near the upper right-hand corner of the site, or click
here.

Now you can see the
Share
button.

Click the
Share
button to browse for the media you want to upload. You can select more than one file, or you can click the Share button again to select additional files.

Archive.org will automatically detect which media collection (movies, audio, texts, or other) your item belongs to, according to the type of the first uploaded file.

You also have the option to click the link to change the file type if needed.

As the file(s) upload, enter the information about your file in the given fields.

When everything is complete, click the
"Share my File(s)"
button at the bottom of the page to create your item page on Archive.org.

I want to add LOTS of individual items to the archive, how do i do that?

If you have a large collection of related items in single media type, like a radio show for example, please contact the Internet Archive. You can email our collections staff at info at archive.org. Please put start your subject line with "Collections:".

Be sure to include the details of your collection; we want to know how many items you have, what format they are in as well as any general information you can give us about the collection.

In general, collection pages are created once the number of uploaded items has reached 50 or more.

How can I report an error for my item?

First, we recommend that you search the Forums. Many common problems have already been answered there, and you'll have an answer much more quickly.

If that doesn't work, you can email info at archive.org. Be sure to include a link to your item's details page. Report the details about the problem you are experiencing - the more details you provide, the more readily we can help you.

How can I make changes to my item?

If you want to
change your item's metadata
(like title, description, file formats and titles, running time, language, etc.), or change the files in your item
(remove files, upload new/more files, rename files, etc.), you can do this using the new “Edit Item!” link. Here's how:

Make sure you're logged in with the account you used to upload the item

Go to your item's details page

Click the "Edit item" link in the lower left box.

Select the "change the information" link

Your changes will appear in 20-30 minutes.

If you have uploaded new files and you want us to make derivative files (smaller, more compressed versions), you will need to do one more thing.

If you would like us to take down an item you have posted, please send an email to info [AT] archive [DOT] org. Please include the exact URLs of the items. Your email must come from the same email address you used to upload the item. This is the only way we can tell that you are the owner of the item.

Can you tell me a bit more about choosing a license?

From the Creative Commons
website: "Creative Commons licenses help you share your work but while keeping your copyright. Other people can copy and distribute your work, but only on certain conditions."

You can choose a license to associate with your contribution and this license will be linked to when users see the details page.

How should I name the files for movies I upload?

Take for example a movie called My Home Video. The identifier (AKA base name) for this movie should be something like MyHomeVideo. The naming convention for the files depends on the encoding.

MPEG-2:
MyHomeVideo.mpeg

MPEG-1:
MyHomeVideo.mpg

DivX:
MyHomeVideo.avi

QuickTime:
MyHomeVideo.mov

Windows Media:
MyHomeVideo.wmv

Real Media:
MyHomeVideo.rm

MPEG-4:
MyHomeVideo.mp4

If you know the bitrate of the encoding (for QuickTime, Windows Media, Real Media, or MPEG-4), please include in the file name as such (using 64 as the bitrate and QuickTime as the format, for example):

MyHomeVideo_64kb.mov

During upload, I get an error message about 'illegal characters' or 'file name prohibited.' What does this mean?

The folder or files that you are attempting to upload have characters in the name that cause problems with the system - so we have designated them "illegal". This includes the following characters in the name:

* ( ) { } [ ] / \ $ % @ # ^ & | < > ' ~ ` ! ? +

In addition, files and folders may not
have spaces in their names.

You will need to remove any of these illegal characters by renaming the file(s) in order for the system to accept your contribution.

What languages are supported by Archive.org? How can I use accented or special characters in my title or description?

What languages are supported by Archive.org?

Archive.org supports all metadata about items in just about any language so long as the characters are UTF8 encoded.

Additional character support for filenames is not an area under development at this time.

How can I use accented or special characters in my title or description?

You can use accented and other special characters in your item text and file titles, but you need to make sure you use the xml-safe code for those characters instead of typing them directly into the forms.

Typing accented characters directly into forms can break the xml for your item, making your files unavailable through the site.

The Internet Archive strives to archive content in open formats that are friendly to long-term storage and access. In addition to affecting long-term storage and access, giving us media in these formats will assure that they are accessible now, since many problems with long-term accessibility such as DRM and propriatary codecs also cause problems today.

However, if you have content that is not available in an open/recommended format (see below), we will still happily archive it. Our systems are not tied to specific media formats and in fact are capable of archiving any type of digital data that can be represented as a file.

Format Recommendations:

We encourage users making contributions to the Archive to create as high quality versions of their media as possible. As we know access is important and not everyone has a high speed connection, we will take these archivable copies and create much smaller version for users with slow connections. Remember, a WAV file may seem big, but it won't be in 5 years. Further, you can always make lower quality files (e.g. mp3s) from higher quality files, but cannot go the other way.

For video we typically recommend MPEG2 (DVD quality), or if you do not have MPEG2, MPEG1 or MPEG4.

For audio we recommend WAV or FLAC (preferably 24 bit).

For text we recommend plain text, xml, or pdfs.

How should I name the audio files I upload?

Take, for example, an audio called My Music. The identifier for this audio should be something like MyMusic. The naming convention for the files depends on the encoding.

MP3:
MyMusic.mp3

WAVE:
MyMusic.wav

Flac:
MyMusic.flac

Shorten:
MyMusic.shn

Ogg Vorbis:
MyMusic.ogg

Windows:
MyMusic.wma

Real Media:
MyMusic.ra

If you know the bitrate of the encoding, please include it in the file name. For example:

MyMusic_64kb.mp3

How can I take my files off the site?

If you would like us to take down an item you have posted, please send an email to info [AT] archive [DOT] org. Please include the exact URLs of the items. Your email
must
come from the same email address you used to upload the item. This is the only way we can tell that you are the owner of the item.

I just uploaded my files, and I got an error message that says there's a problem with my metadata - but I haven't added any metadata yet!

When you create an item, we "check out" a directory for you to upload files into. When you're done uploading, you "check in" the directory (by cicking a link on the check out page, or clicking the "click here when done" icon).

Checking in an item lets us know you're done uploading, and the first thing we do is back up your files to a second server (so we'll have two copies of everything). Sometimes, when it's taking longer than usual to complete this backup, you'll get an error message that says there's a problem with your metadata. If you wait a little while (usually just a few minutes, but occasionally longer), you should be able to continue the upload process without any trouble.

If you uploaded metadata with your files, or you've gotten this error
after
you've added metadata (title, description, file titles, etc.) then you may have a problem. Usually an item breaks because you used special characters
that broke the xml files for your item. Please feel free to use the link on the error page to report the problem to us and we'll try to help you fix it.

Starting in 2011, the Internet Archive began automatically retrieving BitTorrent files uploaded into most Community collections.

Uploading a Torrent provides a convenient way to upload many files or large contents, provided seeds (including webseeds) are available for the Torrent.

How to prevent an Archive Torrent from being made

Internet Archive BitTorrents are automatically made for community-contributed items in many collections, and automatically updated when item contents or metadata change.

If you prefer that your item
not
have an Archive Torrent made for it; or that items within a collection you maintain do not, you can prevent Torrents from being made by including the following metadata tag in your item:

noarchivetorrent=true

Note: adding this tag does not remove existing Torrents, those must be removed using the Item Manager item file management tool.

For instructions on how to edit an item or collection's metadata, see the FAQ
Uploading Content.

Can I download only part of an item using an Archive BitTorrent?

Yes, almost all contemporary BitTorrent clients allow you to select which files included in the Torrent are downloaded. And even when you download only one or some files, you get the speed advantages of using the format.

Many show a list of the files contained in the Torrent, and both folders and individual files can be selected or deselected both before, and during, download.

It is recommend, in fact, that you deselect the top-level directory within the Torrent named
._____padding_file
if there is one, as this contains unnecessary (empty) Internet Archive padding files.

My Torrent download never completes?

Most likely, you have an out-of-date Torrent for the Item you are trying to download. The first thing to try is re-downloading the Items' Torrent, and trying again.

Torrents for Items on the Internet Archive can become obsolete when the Item the Torrent is for changes. In that case, some or (more rarely) all of the files within the Torrent will fail to download completely.

This is because our Torrents rely heavily on webseeding (download directly from our servers, when no peers have the files you are seeking). When files on our servers have changed since the Torrent was made, they will not match expected 'piece hashes';
some BitTorrent clients (e.g. Transmission) will attempt to re-download file pieces from changed files over and over, forever, assuming there was an error in transmission, when in fact the file has changed.

Torrents that never download at all most likely are the result of a different problem, lack of client support for Getright-style webseeding.

My Torrent download never starts?

It's worth mentioning that some BitTorrent clients take a very long time to begin downloading when relying on webseeding
(a common requirement when using Archive BitTorrents). At times downloads can take upwards of several minutes to start.

We're not sure exactly why; we suspect those clients exhaust all other options, such as DHT, before falling back on webseeds. (We have observed this behavior with Transmission.)

If you download an up-to-date (current) Torrent from the Archive, and it loads into your BitTorrent client, but download
never
begins, the most likely cause is that you are using a BitTorrent client that does not support Getright-style webseeding.

Our Torrents rely heavily on webseeding (download directly from our servers, when no peers have the files you are seeking).
Some BitTorrent clients (e.g. rTorrent) do not support Getright-style webseeding, and will not be able to download un-seeded Internet Archive Torrents.

At the moment, the only solution to this problem is to use a different client.

Another possibility is that your Torrent file is out of date, because the Item has moved to a new server, and your client does not support redirection of our canonical webseeding URL (and no tracked or discoverable peers are seeding the Torrent).

In this case, the problem can be solved by re-downloading the Torrent file.

How do I tell if a Torrent is being seeded?

Current seed and leech counts are displayed for each Archive Torrent on the relevant Item details pages, in parenthesis next to the Torrent
link. These values are cached for five minutes or so, and because clients do not always update our trackers regularly, they may be somewhat out of date.

The number of seeders is shown first, and the number of leechers (downloaders without the complete Torrent) second. The seeder number includes 'webseeds,' however, which are only usable by BitTorrent clients which support Getright-style webseeding.

Why is the Torrent link for an Item lined out (Torrent)?

While an Item is being updated, its Torrent link is temporarily
disabled and shown as Torrent.

Changes to an item usually render any existing Archive BitTorrent for it obsolete. Attempts to download obsolete Archive Torrents will usually fail, as described here:
My Torrent download never completes?. (Technically, the problem is that when files within an Item change, they can no longer download correctly via webseeding because the piece hashes for updated files change).

The Torrent link will return to normal when the Item finishes updating and the torrent is updated. The Torrent link may be unavailable for a few minutes or a few hours depending on the size of the Item and how busy the Archive processing cluster is (in very rare cases, it might be disabled for a day or more).

Note: obsolete torrents will continue to be tracked by Archive trackers for some time, but will only be retrievable when seeded by peers who have downloaded the referenced version of the item.

What are peers, seeds, leechers, and snatches?

BitTorrent is a peer-to-peer file-sharing protocol facilitated by centralized trackers. The Internet Archive runs several BitTorrent trackers to allow for peer discovery.

Archive trackers track (but do not log or otherwise record) which peers have pieces of which Torrents; real-time statistics are summarized on
tracker hotlists
for each of our Trackers.

Internet Archive tracker statistics of interest include:

Peers: the total number of clients known by the tracker to have pieces of a Torrent, i.e. the sum of seeds and leechers.

Seeds: the number of clients known by the tracker to have
all
of the pieces of a Torrent available, i.e. those which have downloaded the entire Torrent but remain online.

Leechers: the number of clients known by the tracker to have
some
of the pieces of a Torrent available, i.e. those currently downloading the Torrent.

Snatches: the number of clients known by the tracker to have downloaded a given Torrent, but which are not currently seeding it.

Note: Internet Archive seeder and peer counts include webseeds; these seeds are available only when using clients that support Getright-style webseeding.

Questions

Forums

How can I make links clickable in my posts?

You may have noticed that some posts have highlighted links in them. Internet Archive forums permit the use of HTML codes. Suppose you want to make a link to the Internet Archive home page, one that looks like this: Internet Archive home page. To do this, you would enter the following HTML code: <a href="http://www.archive.org">Internet Archive home page</a>.

How can I format text in my posts

Since the Internet Archive forum system accepts HTML codes, you can make text bold, italic, underlined, or even colored by using normal HTML codes. See WebMonkey
for a list of HTML codes.

How do I subscribe/unsubscribe to a forum email list?

Next to all forums, you will see a small envelope. When logged in, you can click on this envelope which will allow you to subscribe or unsubscribe to any forum.

SFLan

How can I connect to SFLan?

With a laptop: Be in the vicinity of a SFLan node. Associate with it: The SSID is sflanNN, where NN is the number of node, e.g. sflan13. No WEP. You'll get an IP number assigned via DHCP. With a house: Contact us at info at archive dot org. (Please include your address and a phone number.) Find out if you have line of sight to another SFLan node, buy a node, and we'll put it on your roof.

I live at 123 Main St at Crossing; do I have line of sight access to a node?

Go to our map at: http://woody.archive.org/nagios/cgi-bin/statusmap.cgi and see how close you are to an existing node. Also you can try something like netstumbler or kismet to look for a SFLan ssid.

What is the cost of a node?

The nodes cost $1100, which includes the price of parts and installation. Discounts are potentially available depending on the location.

How can I get a node?

Send an email with your name, exact address and phone number to info at archive dot org. Be sure to write "SFLan node" (or something similar) in the subject line. The information will be passed on to our fantastic installation team who will contact you.

Most of our nodes are composed of two radios, but some have three. The components are in a weather tight box with a four foot coax cable and two antennas attached. The whole unit is mounted on your roof (generally) on a pole. There is a picture of our lovely 5'3" spokesmodel holding one here: http://www.archive.org/iathreads/uploaded-files/AstridB-PICT0017.JPG

What are the power requirements of a node?

A node takes on average 5 watts.

What are the connection characteristics of the network?

There are no average characteristics, but 2MBs shared among 20 or so people would be an example.

What is the percentage of uptime?

SFLan is an experimental network, so the uptime varies. Right now uptime averages around 90% or more.

What about IP addresses?

SFLan uses real, routable IP addresses. These are usally given out dynically via DHCP. The nodes themselves use static addresses. We can also assign static addresses for servers. For the techies: We use tunneling, layer 2 and layer 3 bridging in parts on the network to make it all appear as a "flat" LAN. There are pros and cons about this approach. It has worked best for us so far. However, it is a moving target, and might change in the future.

I still have more questions, what should I do?

SFLan is a work in progress. If you have more questions, try the SFLan forum. If you still need help, write to info at archive dot org.

Questions

Report Item

How do I report that there's an issue with an item?

The Internet Archive (Archive.org) is a nonprofit library that preserves digital cultural artifacts, and provides online access to over a million users a day with the goal of universal access to all knowledge.

To report an item which violates the Internet Archive's
Terms of Use, please send an email
with the URL (web address) of the item
to info -at- archive.org

The Internet Archive follows the
Oakland Archive Policy
for Managing Removal Requests And Preserving Archival Integrity. (When reviewing the Oakland Archive Policy, please note that information about requests coming from webmasters is information to assist with archived websites in particular.) For more information, see
http://www.archive.org/about/faqs.php#Rights.

How do I report that something's wrong with a book?

If you see an error with a book that the Internet Archive has digitized, we'd appreciate knowing about it!

Please send an email
with the URL (web address) of the book, and description of the problem, to info -at- archive.org

In some cases, you may know of
alternate
information about a book that is supplemental to the library bibliographic record. (For example, a new, more modern transliteration of an author's name.)

To share additional information like this, you may wish to post it using the option to write a review of a book.

Some changes to our system, to individual items, or to collections can take a day to appear on Archive.org. If you're experiencing a problem with an item, we recommend trying again after a day. Often the issue will then have already been resolved.

How can I take my file off the site?

If you would like us to take down an item that you have uploaded, please send an email to info -at- archive.org

Please note that you need to include the URL (web address) of the item.

Your email must come from the same email address you used to upload the item. This is the only way we can tell that you are the owner of the item.

As always, if you write in, please be sure any spam filter you have is set to accept email from @archive.org.

Questions

Rights

Can I use this ____ for ____ ?

Internet Archive does not itself seek to limit use of its digital materials. However, we cannot give ironclad guarantees as to the copyright status of items in our Collections and cannot guarantee information posted on items’ details or collection pages regarding copyright or other intellectual property rights. Our terms of use (http://www.archive.org/about/terms.php) require that users make use of Internet Archive's Collections at their own risk and ensure that such use is non-infringing and in accordance with all applicable laws.

The person who uploads an item often provides information related to use rights, either by way of directly entering it in the description field or by selection of a Creative Commons license. The latter, if included by the uploader, will be viewable via a Creative Commons
logo on the details page, which serves as a link to a description of the specific type of license that the uploader has assigned.

One way to attempt to contact an uploader about information that they have posted is to post a review to the item.

A link the Terms of Use
for Archive.org is at the bottom of each page.

How can I contact the person / group who uploaded an item?

Internet Archive is unable to release any contact information for patrons. However, it may be worth your while to post a review for the item in question - this automatically contacts the uploader's account, notifying them that their upload has been reviewed. You could pose queries/requests for information therein.

Questions

Archive-It

What is Archive-It?

Archive-It
is a subscription service that allows institutions to build and preserve collections of born digital content. Through the user-friendly web application, Archive-It partners can harvest, catalog, manage, and browse their archived collections.
Collections are hosted at the Internet Archive data center and are accessible to the public with full-text search.

Why would I subscribe to Archive-It instead of using the Wayback machine at Internet Archive?

Partners to this service can create distinct Web archives called "collections", containing only the born digital content they are interested in harvesting, at whatever frequency suits their needs. All collections are full-text searchable. The collections created with Archive-It can be cataloged with metadata and managed directly by the partner. The Archive-It service maintains a minimum of two copies of each collection online, a primary and a back-up copy.

How frequently can I archive Web sites?

Archive-It is very flexible: you can harvest material from the Web using ten different frequencies, from daily to annually. Partners can select different crawl frequencies for each chosen URL. Additionally, your institution can also chose to start a crawl "on demand" in the case of an unforeseen spontaneous or historic event.

Who gets access to the collections created in Archive-It?

By default, all collections are available for public access from the main page at
www.archive-it.org. However, a partner can choose to have their collection(s) made private by special arrangement.

How can I search the collections?

Archive-It provides full text search capability for all public collections. You can also browse by URL from the list provided for each collection. The public can browse and search collections by partner type or collection from www.archive-it.org.

What types of institutions can subscribe to Archive-It?

Archive-It is designed to fit the needs of many types of organizations. The 190+ partners include state archives and libraries, university libraries, federal institutions, non government non profits, museums, art libraries, and local city governments.

Who decides which content to archive in Archive-It?

Partners develop their own collections and have complete control over which content to archive within those collections.

Where is the data stored for Archive-It collections?

All data created using the Archive-It service is hosted and stored by the Internet Archive. We store two copies online and are working with partners to have redundant copies in other locations. Partners can also request a copy of their data for local use and preservation to be shipped either on a hard drive or over the internet.