v2.0 (02/10/1999) - complete rewrite
v2.01 (02/18/1999) - minor fixes
v2.1 (03/12/1999) - Explicitly stated how to upload
The purpose of this document is to specify how to properly upload a file for processing on the Hobbes archive (henceforth known as Hobbes) so as to minimize the time for it to be processed while maximizing the accuracy of the archival.
This document may be copied and distributed with the permission of the author as long as no modifications are made. Suggestions and feedback are encouraged and appreciated.
Hobbes is an archive originally intended for freely-distributable OS/2 programs (kept in /pub/os2). However, in addition to that, the following types of files are accepted:
For more information, a complete listing of all the directories is available for browsing.
First, all files must be compressed in .zip format, except for the following:
Multiple-archive distributions are accepted, provided that some of the archived components are optional (for example, binaries and source code, or compilers with optional frontends).
In the /pub/incoming directory is a file called README.TEMPLATE which is a template for describing files in the incoming directory. Along with every upload, you must include a .txt file with the same base name as the archive (for example, abc123.zip needs abc123.txt, def456.mp3 needs def456.txt, and so forth). It should be filled out in a normal text editor, with the default values removed if they are not used. This file is a plain text file with the following fields, all of which are required unless otherwise specified and must start after the : or ? character, on the same or next line. (Note that this file uses the term "program" to mean "program or file;" a multimedia file, for example, is still referred to as a "program" in this context.)
Most of this file (except for the placement directory and replacement fields) is automatically-processed by a program, which tries to be intelligent about mapping a file to its components. However, this isn't always possible, particularly if you use a format other than the one given. The database is updated twice daily, so a few hours after uploading the file (12 hours at the most), please check the database, the extended information (the 'More Info') link, to be sure that it appears how you want. Please note that, as a necessity, no formatting is preserved in the file, and the long description is truncated to the first paragraph.
Please always use the latest README.TEMPLATE file. It can change from time to time, and older formats may not necessarily be supported.
Uploaded files are only accepted via FTP. I cannot accept submissions via email, for a variety of reasons, and I will not be implementing any web-based upload facility. Do not email files to me (even the .txt files); it's very easy to find an FTP client (OS/2 even comes with two of them, and there's plenty on Hobbes), and it is much simpler for everyone involved.
There are a variety of reasons for a file needing to be re-uploaded; for example, an upload may have failed, or a .txt file may not have produced the correct results. If this is the case, you will have to rename the old file (most FTP clients, including the ones which come with OS/2, let you do this) to filename.delete and then reupload the file under its original name. For example, if abc123.zip fails on upload, rename the old one to abc123.zip.delete and then reupload abc123.zip. If the upload fails a second time, you can rename the file to any variation of filename.delete, such as abc123.zip.delete2. The same goes for textfiles.
If you're uploading a new version of a file still in /pub/incoming, please treat the previous version of the program and its filename as a failed upload. For example, if you upload abc124.zip while abc123.zip is still in /pub/incoming, please rename abc123.zip and abc123.txt to abc123.zip.delete and abc123.txt.delete, respectively. This makes things much easier for the archiver.
After it is uploaded, the file will sit in /pub/incoming until it is either archived or rejected. If it is archived, it is in the archive until the file is superceded by a newer version or the archive itself suffers a dire fatality. If it is rejected, it is usually for one of the following reasons with the listed consequences.
This is the most common reason for rejection. If there is an email address available in the textfile, the uploader will receive a form letter stating the nature of the rejection and giving general instructions on rectifying the situation. The rejected file will be stored in /pub/incoming/delayed until either a week has passed (at which point it will be deleted) or the uploader fixes the situation. Normally, uploading a corrected .txt file will automatically move the file out of delayed on the next database update (12 hours at the most). Its listing within the delayed directory will include the reason for its rejection.
When this happens, the uploader should receive a form letter indicating that the file was bad for whatever reason. The file will be deleted, and the .txt file placed in the delayed directory, and will be automatically moved back into the incoming directory at the next database update after the file is reuploaded (or until a week has passed, at which point it is deleted). Uploading the .txt file again shouldn't be necessary, unless it is not in the delayed directory.
Although Hobbes is an international archive, it is run in America, by English speakers, and most of the users speak English. As such, it is inappropriate to assume that the archiver will understand German/Spanish/Swahili/Esperanto/Basque/etc., as is it inappropriate to make such assumptions about other users. Having a German description along with a short English description is appropriate, but in general it's better to describe the program in English and, if the program itself is in a language other than English, put the language in the short description. For example,
Archive Filename: abc123.zip
Short Description: Learn to count numbers (Spanish)
Long Description: Esta programma es muy bien si usted querra que usar los numeros que son muy dificil.
This is generally due to a DOS or Windows program unrelated to OS/2 being uploaded to Hobbes. It can also be due to inappropriate content (Hobbes, being run by the New Mexico State University, cannot carry any sexually-explicit material, among other things). The uploader will get a message stating the problem, and the file and .txt file will be deleted. If the uploader is able to correct the problem in some way (such as by removing adult-oriented content), they can feel free to upload the corrected file.
Hobbes is for freely-distributable material only. Pirated programs, music, and images are specifically prohibited. If such material is found on Hobbes, it will be deleted, the uploader's ISP will be contacted, and, depending on the seriousness and repetition of the offense, possibly temporarily banned from making FTP connections to Hobbes. (For example, some ISPs have been banned due to users uploading a homepage to /pub/incoming and attempting to make the index of /pub/incoming appear to be that homepage, thus also confusing users.) Note that attempts to upload pirated material, as well as attacks on Hobbes, are taken very seriously and as many measures as necessary will be taken to track down the offending user. No remorse is given to people who "didn't know better" or "thought it'd be fun[ny]."
If you, as a user, discover such material on Hobbes, please don't hesitate to email the archiver so the situation can be taken care of soon enough to minimize damage.
Regarding such happenings: I have had many users suggest to me that, in order to prevent future attacks from happening, I should set /pub/incoming write-only. I refuse to do this for several reasons:
There are multiple answers to this. There are scripts which run every 12 hours (at 05:23 and 17:23 Mountain time as of this writing) which automatically clean up /pub/incoming, set the descriptions of new uploads (wherever possible), and rebuild the static HTML pages. The pages you view through the CGI programs, however, are always current as to the state of the database, which is modified directly by the archiver as files are processed. The scripts for cleaning up /pub/incoming are also occasionally run by hand to get a group of new files in an archivable state sooner.
First of all, there are no "robots" running on Hobbes, just the scripts (as mentioned above). All of the email-generating offenses are done by hand. So as to save time, energy and typing, there are a number of form letter-generating programs which the archiver runs to send out a quick but lengthy and repetitive message to an uploader who was uninformed regarding the rules. The archiver is sometimes wrong, of course, and a polite email in response to the form letter works wonders.
I have received a number of complaints regarding my "robots" being too stupid to figure out what a text file meant or what directory a program is supposed to go into. No, chances are, it was me who was too stupid to figure out what a text file meant; if I couldn't figure out what someone means, chances are a mere program wouldn't be able to either.
Yes, it's right here.
A custom-written hard-coded database and indexing system written in C++, with all flexibility sacrificed for the purpose of speedily indexing a large FTP site.
Although I believe in the Open Source Movement, I want to maintain complete control over the system, and frankly, I'd be too embarrassed to let people see the source. And anyway, it's not intended for a general case; it's written and hard-coded for a specific hardware platform, namely an old RS/6000 (about the processing equivalent of a 486/33), and it'd be only marginally useful on anything else. It'd also not be for the faint of heart, when it comes to trying to configure or extend it to fit your needs. Also, there's probably niggling legal issues, since it was developed for NMSU on NMSU payroll, and is thus technically NMSU property.
If you have any questions, suggestions, or warm fuzzies regarding these policies, feel free to contact the archiver. Intelligent discussion is appreciated.