SiteMap
 USMS History & Archives Committee

The USMS Archives on a CD

(reminders 11/20/03)
  1. The shortcut we build Sophie will need to know (a) the location to store the data file, and (b) the drive letter of their CD drive. The two applications will need separate shortcuts as they'll have to point to different places on the CD.
  2. Don't forget to include any icon files on the CD that you want to use on shortcuts, including any on the CD itself. Such icons will also have to be copied to the desktop with the shortcut itself.
  3. If we had the time & money, we could write a routine that would create the shortcuts for them. We don't right now, but that can be an add-on later.

(observations 11/18/03)
to get back to Derek's form, either minimize or close the browser, or you can click on the bottom task bar.
instructions for using the browser can be put in the browser (Carl can do this by editing FindImage)
we should change "usms logo 80.bmp" to "usmslogo80.bmp"

(reminders from Davin, 10/6/03 & 10/13/03) I show two files that are duplicated: \swimgold\pix\FederalWay.jpg \swimgold\pix\PBP16thLoggerheadDamonHiggins.jpg If the filenames are not unique, the editing process will be compromised. Can you ensure that the filenames are unique when building the file?

Don't forget to enclose the items in [;7] of the main database. You used to use it for a single ID, but now we're going to allow a nested vector of those ID's. So a single ID would look like: {ravel}{enclose}'XXXY9Y9' and one without an ID would be: 0{rho}{enclose}''

Also, in column [;7], you seem to have several items that appear to be file names instead of swimmer ID's. We'll want to make sure those are cleaned up.

In column [;1], watch out for using file names with "~" in them (short, old-fashioned, DOS file names). Try to use standard (long) file names everywhere instead, because Windows likes to change the names of short file names on a whim, which would cause the images to be inaccessible.

The 11th column of the SID table is the old SwimmerID and is necessary to get at anyone's information in USMS archives which will be stored in their entirety on the CD.

(Davin specs 9/21/03) (note that SEX values are MW, not MF) The "database" will now have two tables (matrices), the new one will be in file component 29 (origin 1). When first reading them in, we'll combine them (on a key field in [;1]) into a single table in memory for searching purposes. (This may cause a few images to appear in the list more than once, so the search routine will eliminate duplicate names before returning matches.) The secondary data will contain only reference information and will not ever be editable (and probably not even displayable as output). The new fields to be stored (with original & merged column numbers) are: [;1/11] SwimmerID (matches one of the swimmers in the picture, not shown on screen) [;2/12] Gender ('M', 'W', or '' ["Either"/default] via Radio buttons) [;3/13] Last Name [;4/14] First Name [;5/15] Middle Initial [;6/16] City [;7/17] State (verify exactly 2 letters [if any] to search on) [;8/18] Zip (greyed out on screen until some data is present in the database) [;9/19] LMSC (see function LMSC_Names) [;10/20] Club

In addition, the application function is somewhat different...

When the application is in read/write (editing) mode, a search brings up a match (if only one) into the screen fields for editing. (Extra search-only fields are blanked out & disabled.) If more than match is found, a list is presented (probably via a ListView:Report) showing the matching picture text (several columns of data may be displayed here), and a choice is made as to which one will be edited. That choice is then displayed as above.

However, when the application is in browse (read-only) mode, the behavior is different. The search/display/edit form is then used *only* to enter search criteria. Matches (output) are not displayed on this form. Instead, simple HTML is constructed (with a little heading information) containing only a list of tags pointing to the (multiple) matching image files (thumbnails with links to full images) directly on the CD. This HTML is then written (using 'TextFile') to the user's temporary file directory on their hard disk (using 'SystemFolderNames') and their browser is invoked (using 'RunFile') to display them (images only) in a separate window. If only one file matches, then show ('RunFile') only that image file in the first place and skip the thumbnails & generated HTML.

Current responsibility breakdown:

Davin will write utility functions to read in the two tables from disk and merge them together into a single search matrix. Davin will also write a utility to search this merged table by criteria (supplied in argument) and return a unique list of matching filenames.

Derek will build all the necessary forms and write the code (handlers) to hook everything together. This includes updating the main file component with any changes if in read/write mode. The workspace will be prepared to be used in a run-time environment, though Carl will have to actually )RSAVE it so it will use his serial number. The code should be kept small and simple where feasible. In the form construction, see if it's feasible to use the technique of identifying the database column (by database name?) with a UDP on each search field on the screen so that the general code need not be hardwired to reference the database. My new {quad}io utilities might be helpful here -- ask about them.

Derek, let's see if we can use the following mechanism to determine whether they're in read/write or browse-only mode. Let's assume that they'll have a shortcut on their desktop or start menu. (We can supply it on the CD and let them copy it to their machine if they want to.) This shortcut will start APL (directly from the CD) with an extra argument on the command line. The argument will be in the form: DB="C:\path\to\their\db\IMGDB.sf" If this parameter is not present upon startup (use 'APL_Parms'), we run in browse-only mode strictly from the CD. However, if the parameter is present, it will point us to the full path name of the database file on their hard drive, and the application should then run in read/write mode. That's the only file name that changes in the application, everything else (including the workspace) still resides on the CD and 'CWD' is used to find it. If the parameter is present, but the file does not exist at that location, copy it to that location from the CD before starting (pre-create directory with 'CreatePath').

Carl will continue to provide the database file containing the master data tables. This way he can supply the extra information in the file that he needs for other purposes. He is also going to need to be doing any re-building of this information in the future, so he better have routines ready for that event anyway.

Carl should probably be producing the final distribution version of the CD himself. He can just )RSAVE the workspace onto it and he can reuse the other startup files & configuration that he developed last year for a similar CD release. He should also include on the CD all the images & thumbnails in his preferred directory structure (matching the names shown in the database file which he should also copy there), already sized and annotated as desired.

(Davin offers the following comments on speed browsing on the CD) For one thing, startup speed might be slow because it's going to have to read the APL interpreter (and any associated support files) off the CD before it even gets to the workspace and database files. If so, is there even anything we could do about it if we wanted to?

Secondly, when we present them with thumbnails, it's going to take the browser time to go find them on the CD and show them on the screen. For one or two they won't even notice, but if there are dozen(s) of matches this could take a while.

Thirdly, since we're using a browser to display our thumbnails/images each time, it's going to have to take time to actually start the browser on their machine. This is likely to only take 2-3 seconds, but it's going to have to happen every time they press find. (A built-in APL control could be left on the screen for immediate re-use, but we can't do that with a browser.)

(Davin specs 9/18/03) Here's a structure definition for our new IMGDB project database file. Let's name the file IMGDB.sf and we'll store it in the same directory as the workspace (let's call it IMGDB.w3), which can be determined by using CWD. Carl will be responsible for creating this file and filling it with data from his old database system. We will eventually add a facility whereby the database file (only) can be copied to a hard drive, which signals that editing of the fields is permitted.

Carl would like the file to have several reserved components, so the first 7 components will be reserved and ignored by the application. The following component (#8 in origin 1) will be the only data component of the file. It will contain a nested matrix with 10 columns and a few thousand rows. The columns will be defined in fixed positions as follows:

[;1] Filename (text, non-editable) The name of the photo file, including full path (starting with "\") and extension, but without a drive letter (assume 2{take}CWD).

[;2] Subject (text) Name(s) of person(s) appearing in the photo.

[;3] Comment (text) Brief text about photo (e.g. date, photo location , page #, etc.)

[;4] Photographer (text) Name of person that took the picture

[;5] Copyright (text) Copyright holder notice (without symbol). May include leading year. (e.g. AP, NYT, AllSport, etc.)

[;6] Text (text, long) Photo description or narrative text.

[;7] SwimmerID (nested vector of text vectors) Zero or more alphanumeric ID's. This is stored as a nested vector of separate ID's. When presented on the screen, show them listed with commas (use ^commalist). When editing, allow input to be separated by any reasonable delimiters (including spaces & commas). ID's will be restricted to letters & digits to help identify separators between ID's and auto-capitalized so that they can be more easily searched (they are case-insensitive) and to improve readability.

[;8] SubjectRelease (boolean, checkbox, tri-state when searching) Have the subject(s) of the photo given their permission for its use?

[;9] PhotographerRelease (boolean, checkbox, tri-state when searching) Has the photographer given their permission for its use?

[;10] CopyrightRelease (boolean, checkbox, tri-state when searching) Has the copyright holder given their permission for its use?

Process: When the user searches the database, they fill in any known information on a blank edit screen. Any non-blank fields are used as search criteria, looking in the corresponding database columns for matches. If no matches are found, display a popup message to that effect. If exactly one matches, display its fields (and picture) on the screen. If more than one matches, present a Listview:Report showing the choices and allowing a user selection.

(Carl comments 9/18/03) We're calling this application "imgdb". To achieve clarity while saving typing time, we'll call the person who will update "imgdb" Sophie. Sophie will copy the CD onto her hard drive (must "install" if use grid control). USMS photos will in the future be entirely managed with this system. All existing documentation will be migrated into imgdb. (Hopefully Sophie will be able to add new photos.) Her experience in browsing will be exactly like everybody else's who is working from CD except that she will be able to edit what she sees. Any batch processing will be done by the Archives Steward (Carl). Images will be arranged on the CD exactly as on the Steward's hard drive. Sophie only copies the application, it uses images on the CD. Sophie will start the process with a shortcut or registry item that will signal the app we are in edit mode. Mac users cannot use imgdb, so they will be instructed to load their browser and start e:\index.htm. Directories of photos by Club & LMSC will be updated for the convenience of Mac users.

e:\autorun.inf will remain just as it is. The rts.w3 that it loads will be revised so that it assumes most people want to see the photos, but giving people the option to load e:\index.htm if they want to browse html pages. We'll assume that e:\autorun.inf will not hurt Mac users and we won't try to emulate that for Mac users.

Photos to include in the new catalogue you are building includes all 3263 those identified at e:\swimgold\committee\photos.htm plus the 136 at e:\usmsPix\ for a total of 3399 not counting thumbnails or original art.

These are our fields (controls):

  1. Filename - edit box non-editable (this is the key field)
  2. Subject - edit box
  3. Comment - edit box
  4. Photographer - edit box
  5. Copyright - edit box (if year is included, it must precede whatever else is in this field)
  6. Text - edit box
  7. SwimmerID - edit box (alphanumeric, optional)
  8. release: Subject - checkbox, Sophie must make sure has all from multiple subjects, we don't worry about that.
  9. release: Photographer - checkbox
  10. release: Copyright - checkbox
  11. picture control (bmp gif jpg png tif emf)
Notes by Carl on the Preliminary CD Created 9/16/03. Davin & Derek should add more notes as we proceed.

The contents of the preliminary CD as as follows: (it has 235mb free space) 


(2)Volume: USMSARCHIVE; 9/17/03 15:46:00 for e:\muckyduck\chart.htm

e:\                 1       6,500     5  9/16/03  11:16:00         

e:\muckyduck\       4   4,511,995   116  9/17/03  15:46:00         

e:\ttdb\            4  61,296,955    66  5/28/03  10:10:00         

e:\Sydney2000\     95  10,181,201  1449  4/11/03  19:11:00         

e:\swimgold\      135 123,124,723 10013  9/17/03  15:38:00         

e:\swimimgs\      133  56,241,347  3672  9/17/03  09:14:00         

e:\swimpix\         5   3,587,219   253  4/24/03  13:49:00         

e:\rts\             1   2,030,363     6  6/23/03  22:45:00         

e:\usmsPix\         3  31,124,360   275  9/17/03  12:28:00         

Total 15855 files 381 292,104,663 15855  9/17/03  15:46:00         

 e:\qkw.ico            766   9/28/01  16:51:04  32                 

 e:\autorun.inf         27   6/12/03  17:05:41  32                 

 e:\index.htm         5622   9/16/03  11:16:27  32                 

 e:\autorunttdb.inf     45   8/31/02  19:16:45  32                 

 e:\autorunimginfo.inf  40  12/19/02  08:53:34  32

Here are some introductory notes I just wrote up for Derek... (9/15/03) (by Davin)

Here's the basic idea. He'll put out a CD containing our application code (& APL run-time files), an APL-file database (several text fields by a few thousand rows/components), and a subdirectory tree of image files (w/thumbnails). We need to optimize and polish for lookup and presentation, but we'll also have a text updating facility.

We'll want a main screen with dual-purpose text fields on it. They'll enter text in some or all of the fields and press "Find". This will scan the database for the record matching that data and read it out. The full information will then be re-presented in those fields and a side window will open up to show the image (from file) associated with that record. They can then press a "Done" button (or whatever we want to name it) that clears everything out and lets them input new search data. (Don't you think this will work better than separate fields/forms for search and display?)

If multiple records match their search criteria, we need to give them a list to select from. I'm thinking that an APL.Grid might work well here, with columns for various short text fields that will help them choose the right one. One column can probably be used to show a thumbnail image, but we may want to fiddle around with speed issues.

Most users will run this from the CD and the data will be read-only. However, we should detect if the database file is on a R/W disk and if so, allow them to update any of the text fields and write the record back to the file.

If we have funds available, we'd also like to provide a database export command so they can send the whole database to a comma-delimited and/or XML file.

I have utilities to do many of the above pieces, so ask before writing something low-level.

This application should be kept simple (and cheap) -- we won't need to build any fancy generalized utilities for instance -- but the lookup/presentation should be fast and glitzy (to impress detractors). I'm thinking about storing the database file as a one-row-per-component nested vector of field values, but pre-loading all the key fields into memory for faster lookup. But if that's going to be a slow process on older machines, we need to come up with another way. (Maybe a secondary [or imbedded] inverted index file?)

Davin

To Davin from Carl (9/15/03)

We're handling watermarks fine now. Next project is to build an image database of our annotation and images with watermarks as needed. It should be on a CD with a thumbnail & "web" image of each. I will use the watermarking facility to create all the web images & thumbnails. Cost must be under $1500. Work must be completed before 12/31/03. (hopefully well before that so I can get CD's out by 12/31/03).

Thumbnail is always in a subfolder called "t" under the folder with the web image. At present documentation is always in a file called imginfo.sf in a subfolder called qksource. You should not expect to change the folder structure of images, but you can change the database storage method. You must have an xml export facility, possibly only for all images, but it would be nice to export documentation for only selected images. We can assume that future annotation will be done with imginfo.w3 and ported into your database. Or you can build the imginfo.w3 capability into what you are creating. You can assume that future annotation work will be done by 1 person who will learn how, though if the current imginfo.w3 facility continues to be used, then we have more flexibility for getting workers to do anotation.

On my CD dated 9/4/03 you should find imginfo.w3 in d:\dv\ Functions will be undocumented (snuffed). To get documented fns, execute "Davin", then execute "8000 FND {quad}nl 3" A list of images is at http://www.usms.org/swimgold/committee/photos.htm I'll send a CD tomorrow or nextday with all images & documentation on it.

The most important criteria for this work is that it must be easy & impressive for lots of people to browse when they want images for marketing or newsletter purposes. Also remember that some of these people will be snipers wanting to make us look bad. Your work must be in RTS on CD (using my RTS id#). I think you can give me a CD to use as source and I can make CD's or get them made by someone else.

©Copyright 1997-2002 USMS.
All Rights Reserved
horizontal line
What's New Page to home page e-mail Page Top