view · edit · attach · print · history

ToMM » Gallery

See also Tag, image/audio file tagging scripts.

This page has a bunch of notes that deal with a web media file application.

Gallery Management Options: Gallery Management Option #1 (-subdir) :: Gallery Management Option #2 (default) :: Gallery Management Option #3 (-samedir)

Application Operations (What the scripts will do): Perl :: PHP :: Database

Gallery HTML Output: HTML Layout :: Stylesheets

Perl Script Operations: Script Initialization :: Script Main :: Script Debugging :: Test Cases

PHP/Perl Object Attributes

PHP/Perl Object Methods: New :: Resize :: Create Index Page :: HTMLify :: LayoutCost :: Movie Frame Dump :: File Backups :: Export/Import :: Streamscrape

Related modules: Snarfazon

Misc: Support Scripts :: Example ToMM Commands :: Audio File Management :: Filetypes Version Numbering System

Gallery Management Options

Gallery Management Option #1 (-subdir)

Each file of type IMAGE and MOVIE will be held in it's own subdirectory with the directory name being the same as the filename of the original image.
- All of the related files and thumbnails will be held in that same subdirectory
- This is to facilitate the moving of images in order to help grouping of related sets of images/movies.

Gallery Management Option #2 (default)

All of the original files are held either in the starting directory (gallery called without an -outpath), or in a common output directory (gallery called with an -outpath)
Create subdirectories for all of the files related to that directory of image files.
- When you want to create a new subset of images that are taken from the original images, you create a new directory and populate it with the new images, re-generating any HTML along the way.
- If you move an original image out of the directory, you should be able to run a script with the --clean option, which will then remove any resized images and references to those resized images and the original image from the HTML.

Gallery Management Option #3 (-samedir)

Create all of the new files in the same directory as the existing files

Directory naming scheme

Name the directory holding all files or only support files after the original file
(optional) replace the file's original extension with a '.dir' label, so you can do find on support directories easier.
- For example, img_0001.jpg could be stored in the img_0001.dir directory, along with thumbnail files and other resized images.
- The PHP webpage scripts would also be able to filter easier based on this schema as well; it would be able to filter out directories with a .dir extension out of any directory counting schemes.
Use other extensions besides .dir to mark support directories
- .hide
- .nolist

Gallery Management Follies

(default behaivor) write new files into a new directory with the same name (minus file suffix) as the original file in the same location as the original file, leave the original file alone
-subdir write new files into a new directory with the same name (minus file suffix) as the original file, move the original file into the new subdirectory as well
-samedir write new files in the same directory as the input files came from

-samedir and -subdir are mutually exclusive, exit with an error if they are both specified
If -inpath is used, for each path passed in with -inpath, write the resized files in the same directories as the original file came from according to which subdirectory option is used (default, -samedir or -subdir)
- -inpath and -samedir will write files into the same directory as specified in -inpath, with no subdirectories being created
- -inpath and -subdir will cause a new directory to be created for each file in -inpath with the same name as the original file, the original file and all resized files will be placed inside of that new directory
- -inpath by itself will cause a new directory to be created for each file in -inpath with the same name as the original file, all resized files will be placed inside of that new directory but the original file will be left alone
If -[same|sub]dir or default is used without -inpath or -outpath, read originals and write resized files to the current directory
If -outpath is used and -[copy|ignore|move]originals is not specified, exit with an error (use -[copy|ignore|move]originals to tell the script what to do about the original source files now that a new output directory will be used)
group these options together in the POD documentation

use -dirsuffix and -nodirsuffix to add/change a directory suffix that would be used for storing files related to the original image

Application Operations

Application Operations: Perl

Create a PHP and/or Perl script that can either view files stored on a webserver as thumbnails, or in gallery mode, one picture at a time in a new window opened by JavaScript.

Thumbnails: the script should check for existing thumbnails for a media type; AVI's and RAW files will have pre-exisiting thumbnail files which don't need to be re-generated for this script.
- add code to the object class for .thm thumbnail files that searches out the original filename, and then adds the .thm file as a child image of the original file.
create a command line shell, where you can read in a set of files and manipulate them; the shell would allow interactive debugging. Make the shell a separate module, that's only loaded when requested on the commandline, ala CPAN.

Application Options: PHP

Create a PHP application that can read in a directory of files and prettyprint, or use the directory structure outlined below.

Any time a gallery folder is listed, the scripts should create and display a count of files (by filetype) and/or subfolders inside of each folder, so the user can see how full that folder is and what type of content is inside of it
- Somedirectory (5 folders, 15 images, 20 audio files, 3 movies, 2.5G in size)
Create "virtual galleries", which are galleries made up of images chosen from other galleries. When a user is browsing a gallery, they can select images from that gallery to be saved in a virtual gallery. When the user brings up that virtual gallery, the image file links for that virtual gallery page are retrieved from the database and used to build the web page for display to the end user.
- realdir/image1.jpg -> virtualdir/savedimage1.jpg
- someotherdir/image2.jpg -> virtualdir/savedimage2.jpg
- anotherdir/image3.jpg -> virtualdir/savedimage3.jpg
- Users can append additional comments for the virtual gallery as a whole, and/or for individual virtual images in that gallery. The additional comments are above and beyond any comments that may exist for the original image in it's original storage location
Build an index.php script, which will display the files stored in the current directory, whether they file metadata is stored in flat files, in the images themselves (no metadata), or in a database. The script would allow users to enter in the following:
- page titles
- page captions,
- aliases for files and directories
- database data if a database backend will be used in that directory.
  - for flat files, the file/directory would have to be writeable by the user the webserver runs as for the user to be able to edit the above information

Database Functionality

database tables will contain an internal version number, which will help with upgrades as the upgrade script can query the database for the version number, and then run all of the updates needed to bring the database up to the latest version the update script supports
For directories that will be using database files for storing information, the following information will be stored:
- gallery caption
- captions for individual files
- file locations
- aliases to files and directories
- access control for who can add/delete/change caption info and view image info. Access control will also be used to prevent images with humans in them from being readable from non-authorized users
- prevent unauthorized access to images by using a file anonymizer, which would map media files on the host filesystem that are to be protected into the database using random strings; once the user requests the file using the random string filename, the PHP script looks up the full path of the file on the filesystem, reads the file and sends the output to the client using the appropriate MIME filetype.
The database can either be hidden (use a dotfile in Unix) or visible as a separate subdirectory inside of the gallery directory
Use an offline/online attribute as part of a directory's database
- create a locking scheme so that when a directory is being updated by the Perl script, the PHP script will redirect to a "gallery offline" page until the Perl script unlocks the directory.
Once databases or text files are created for gallery directories, they can then be harvested for search keywords, which would enable searching the gallery for images based on the search keywords
database files can also be used to tag user's favorite images, so that the user can then call up their own custom gallery filled with images that they have selected, with the database storing the links to all of the images and building the custom gallery page from the database

Gallery HTML Output

HTML Layout

There's a diagram for cascading stylesheet (CSS) classes located in $CVSROOT/mediatag/docs/gallery_css_layout.sxd. This also has some information on how many images could be displayed for a specific screen resolution, which is in line with the Basic View Layout below.

See the htmlify() method below for notes on how styles will be applied to block elements in the HTML output.

Todos

steal the icons for different media types from Enlightenment, pull them down off of the elive CD

Basic View Layout

have the thumbnails on one side of the page, with the currently viewed image in the center of the page
Minimal file metadata for thumbnail images (no room)
More metadata displayed for the currently displayed image
- Thumbnails on the left/right hand sides of the page will be an issue when thumbnails for panoramic images are displayed
  - possibly resize them *really small* so that they fit widthwise with the rest of the thumbnail images in the thumbnail image box
- user will not be able to select how many thumbnails are viewable at a time, instead they will be able to choose what browser size and screen resolution their computers are currently using

Gallery Layout

display nothing but thumbnails across the screen
- Minimal file metadata displayed (no room)
- LayoutCost will need to be used to make sure that the page is generated with no gaps or holes in the gallery
- new window opens when user clicks on the thumbnail image or metadata
- user can select how many thumbnails will be displayed on a gallery page
  - default is 9 thumbnails, 3 rows of 3 images

Horizontal Bar Layout

display files and some file metadata inside of a colored 'bar' that stretches across the width of the page
file bars created down the entire page
- user can select how many bars to display per page
- new window opens when user clicks on the thumbnail image or metadata
- default is 9 thumbnails, 9 horizontal rows down the page

Stylesheets

draw a diagram of all of the elements that could show up on a page. Make each distinct element it's own layout class
- Diagram located in $CVSROOT/mediatag/docs/gallery_css_layout.sxd
- Give each media file type it's own style, that can be inserted into the output of the page when it is rendered
TODO add the stylesheet style class name to each object, collect the styles via Plugin.pm, so when the output HTML is rendered, the styles can then be added to the final output
Use the file's mime type as the name of the stylesheet; EXIF outputs mime types for image files
See the htmlify() method below for notes on how styles will be applied to block elements in the HTML output.
See the Inkscape homepage for examples of how to change the hover of links in a specific area while leaving links on the rest of the page alone (I think it involves combining CSS selectors for a CSS block so that you need both selectors for that style to be applied to that block)
See the OkCupid stylesheet for examples of how to set hover/link/visited tags
See WebCollab for good examples of changing the colors of form fields (text entry boxes)

Gallery Script Operations

Initialization

Start script and initialize objects/variables using AppConfig Perl module
- FIXME Decide when and where $Config needs to be available. It most likely doesn't need to be passed around everywhere.
- Non-critical modules (Log::Log4perl, PerlMagick, DBD::SQLite) could be wrapped throughout all of the modules so if they fail to load, the program can still run without them.
parse commandline options ($Config->args(\@ARGV))
set log levels for different modules based on options passed in on @ARGV
1. Default to one of the gallery management options: individual file-directories, shared files with external directories, or everything in the same place
  1. --clean cleans extra files if the originals are missing (they have been moved/deleted, for shared files with external directories)
2. If a media filetype module has multiple internal Perl modules or external binaries that could be used to produce the same results, allow the user to change the preference order on the command line;
  - For example, JPEGs could be read with rdjpgcom, ImageMagick, file, or Image::ExifTool; which method should be used first? How to check all of the methods to make sure they are usable?
find and load plugins
1. Create a new object of class Plugin. This object will hold all of the file types and what modules they belong to
2. call plugin's new() method to create, then call register() to register the plugins for different media types.
  - Plugin object holds a mapping of file types and file magic with the modules that handle files of that type
3. register() searches for Perl modules in the Plugins directory, then tries to use() them.
4. if the use() succeeds, query the plugin's filetypes, magic strings, and external binaries and add it to the master list stored in the Plugin object (inside of %_types, %_magic, %_binaries)
- See also the new() method for more info on how the method acts with and without a filename when it is called

Main Script

if a pre-execution backup is requested, it will be done now (-backup_file <filename>, -backup_with [tar|tarbz2|targz|zip] and -backup_exclude *.[zip|tar|bz2|gz])
gallery.pl will then read all of the directories listed in the $Config->path array
- use File::Find::Rule on the --path arguments to build a list of files
- assume $PWD if no --path is used
For each file that is found
1. instantiate a new object with the type found in the Plugin->filetypes hash.
- See also the new() method for more info on how the method acts with and without a filename when it is called
- When the physical file is read by the new() method, the file object will determine the following attributes along with any attributes that are specific to the original format of the file (example: JPEG files will have EXIF tags read and stored inside of the file object):
  - filesize
  - permissions, which is the $mode value from the stat() function call. You need to AND the result to get human-readable permissions
  - user/group ownership
  - file times (mtime, ctime, atime)
  - image dimensions (images only) FIXME what other Perl modules or external binaries could be used to verify image sizes; hint: EXIF data could be changed so that the image size info contained in the EXIF data could become inaccurate over time
  - audio file length (audio files only)
  - file metadata (EXIF/tEXt tags for images, ID3/metaflac data for Audio files)
1. add the new object to an array of media file objects stored in the main program
perform operations on that list of media file objects based on the options used on the command line/in the configuration file
if requested, output the results into one of the following:
- HTML
- Plain text
- add a $Config option so that you can output the command used to generate the output inside of the output (as comments in HTML, or as some text offset from the rest of the text at the end of a plain text file), so you can re-generate the exact output again if desired

Script Debugging

WARN will be used for the following:
- files/directories that are not writeable
- existing files for -[move|copy]original
INFO will be used for the following:
- resizing of images and writing the new image to a new file
- writing new HTML files
- writing of new database files
- deletion or saving of GIF files when a new PNG file is created
- totals of the number of records added to a specific database
- totals of the number of files added to the entire gallery
DEBUG will be used for all other messages
- totals of each directory
- adding individual records to an individual database
- not resizing images because the resized image exists and --redo is not set
logging will be turned off when run in --quiet mode

Script Testing

Adversity testing should also be made a part of these scripts, so if someone else were to pick them up and try to use them, they would actually work.

Command line testing

add multiple -output options, what does AppConfig do?
test different -size values, add logic to handle problems with funky -size values

Filehandling

Set -outpath and verify the script exits if you try to write to a directory you don't have permissions to
Set multiple -inpaths and verify the script exits if not one of the input paths is readable
Try parsing filenames that contain multiple period characters, spaces, tabs, other non-printable characters (this.is.some.file.name.jpg) to see if the regular expression parser for file suffixes is solid.

Gallery Script Attributes

_SUPPORT_DIR
- boolean flag for telling the script that this object does or does not want/need a support directory created to hold files that are derived from the original file
- This flag is ignored when the script is called with -subdir, as all files are moved into separate subdirectories anyways (subdirectories that carry the name of the original file)
_FILETYPES
- List of file extensions that this object can handle

Gallery Script Methods

Script options should be modifiable via the command line. For example, --jpegopts should allow you to pass options to the JPEG module that the module will use when running external programs. Another option could be to use a --jpegbin parameter for specifying the specific binary to use when working with JPEGs. Examples would be perlmagick, netpbm, jpegprogs

New Media Object

if $pluginfile->new() is called with a filename, the object is created using that filename and the object reference is returned (current Media::Plugins::File::set_file() behaivor)
if $pluginfile->new() is called with undef, the filetypes the object will handle is returned instead (current Media::Plugins::File::register() behaivor)

Resizing an image file

For Canon RAW images, convert to JPEGs using dcraw
1. Read the output JPEG, determine the image orientation and rotate the image to match the EXIF data
Convert images to other output sizes as requested, as well as 160x100 thumbnails for viewing in a web page

For resized images, one of the following formats can be specified:

Append the long size to the filename so that an 800x600 pixel copy of somefile.jpg will become somefile.800x600.jpg.
Append the abbreviated size to the filename so that an 800x600 pixel copy of somefile.jpg will become somefile.8x6.jpg.
Append the percentage ratio of the new file's size in relation to the original file's size. If the existing file name is somefile.jpg, the new file that is the 50% reduction of the original file could then become somefile.50pct.jpg.

Appending image sizes to the original filename will help with filename collisions, as well as making human-readable filenames for other tools to parse. * GIF files will be converted to PNG files automatically, and any gallery pages that are created will reference the PNG files only. The GIF files can be deleted once they are converted.

Ignore all of the above, ImageMagick uses the size parameter as the maximum dimensions of a resized image. For example, if you specify 400x300 (400 pixels wide by 300 pixels high) for the resize() function, then ImageMagick will resize a landscape image to 400w by 300h, but resize a portrait image to 200w by 300h, as this matches the original resize specification. So you should use just one value for the resize parameter, and double it's usage with ImageMagic, so that way you know that the image on it's longest side will always meet the maximum size originally specified by the user

Creating an index page

create an HTML index page that lists each image, some stats about that image, and the different filesizes for each image (full size, reduced sizes).

HTMLify

when htmlify() is called on a file object, the object will return the HTML representation of the object, including links to all children images and stylesheet classes
htmlify() method knows about an object's stylesheet class, as it can derive it from the object's mime type.

Layout Cost Function

returns the number of 'spaces' or 'slots' that a file will need in any gallery display. Most images will need just one space, but things like panoramic stitched images and movies will need more spaces to display the thumbnail images in
- movies will be 3 images, snapshots of 25%, 50% and 75% viewed in the movie file
- panoramics will be 1 image, but that image will most likely take up the whole width of the page

Backing up Files

createbackupfile - opens a file for backing up
addfile - adds a file to an already open archive
closebackupfile - closes an existing backup file

Streamscrape

scrape URLs ? out of different types of metadata files (ASX/QT)

Mplayer Movie Frame Dump

Don't know what all those movie files are? Dump frames from each file!!

mplayer -ss 00:05:30 -frames 1 -vo jpeg sam_kineson.avi
Maybe determine the length (in hours:minutes:seconds) of each movie, and dump frames 25%, 50% and 75% into each movie
-dumpframe 25% -dumpframe 50% -dumpframe %75

midentify shell script:

 #!/bin/sh
 mplayer -vo null -ao null -frames 0 -identify "$@" 2>/dev/null |
    grep "^ID" |
    sed -e 's/[`\\!$"]/\\&/g' |
    sed -e '/^ID_FILENAME/ { s/^ID_FILENAME=\(.*\)/ID_FILENAME="\1"/g;
  }'

Export and Import

Dump the individual files and database data that would be needed to import the gallery onto another machine. Export/Import could also be used to move files amongst directories on the same machine via the Perl command line tools

Don't delete old files until the import has successfully completed

Misc.

Support Scripts

test.pl will verify that the user has the correct Perl modules that the gallery application will need to run. This information can be collected by grepping the Perl scripts and modules for the 'use' compiler directive
- test.pl/check.pl can also test the database versions used, by querying the database when it is run.
- Modify the perl_swiss_army_knife.pl script to do the module testing, that way the script can then output debugging information as needed/requested
- Check for needed support binaries as well
  - dcraw
  - rdjpgcom
  - mplayer
move imageinfo.pl from $CVSROOT/scripts to this module
- modify the script so it will compare files between one or more source directories and one or more target directories
- use the script when running tests for verification that things are doing what they shoud

Example gallery.pl Commands

 perl gallery.pl -path ~/imagetest/ -debug all 2>&1 | less

 perl gallery.pl -inpath ~/imagetest/ -outpath ~/outtest/ 
 -debug all 2>&1 | less

 perl gallery.pl -path ~/Photos/imagetest/ -debug all 
 -outpath ~/Photos/outtest/ -size 400x300 -size 800x600 2>&1

 perl gallery.pl -path ~/Photos/imagetest/ -debug all 
 -outpath ~/Photos/outtest/ -size 400x300 -size 800x600
 -dumptags 2>&1

 perl gallery.pl -path ~/Photos/imagetest/ -debug all 
 -outpath ~/Photos/outtest/ -size 400x300 -size 800x600 
 -copyright "Copyright (c)2005 Brian Manning" 2>&1

 perl gallery.pl -path ~/Photos/imagetest/ -debug all 
 -outpath ~/Photos/outtest/ -size 400x300 -size 800x600 
 -copyright "Copyright (c)2005 Brian Manning" -overwrite 
 -copyoriginal -dumptags 2>&1 | less

 perl gallery.pl -inpath ~/Photos/imagetest/ -debug all 
 -outpath ~/Photos/outtest/ -size 400x300 -size 800x600
 -copyright "Copyright (c)2005 Brian Manning. All rights
 reserved. Use only with permission." -overwrite 
 -copyoriginal 2>&1

 perl gallery.pl -inpath ~/Photos/imagetest/ -debug all
 -outpath ~/Photos/outtest/ -size 400x300 -size 800x600
 -copyright "Copyright (c)2005 Brian Manning. All rights
 reserved. Use only with permission." -overwrite
 -copyoriginal -subdir 2>&1

 find . | grep "[A-Z][A-Z][A-Z]$" > uppercase.txt
 perl ~/cvs/scripts/file_renamer.pl -f uppercase.txt -l
 find . -name *.8th.jpg -or -name *.half.jpg | xargs rm

 16Jan2006 manzana

 perl gallery.pl -inpath ~/Files/gallerytest/in 
 -outpath ~/Files/gallerytest/out -size 400x400 
 -size 800x800 -copyright "Copyright (c)2006 Brian Manning.
 All Rights Reserved. Use only with permission." 
 -overwrite -copyoriginal -debug all 2>&1

Audio File Management

Tag/Gallery Filetypes

PNG - tEXt blocks
- Perl - http://mc2.vicnet.net.au/home/newtons/web/png/png-text-pod.html
JPG - EXIF data
- PHP - http://www.php.net/manual/en/ref.exif.php
- PHP - http://pel.sourceforge.net/
- Perl - http://www.sno.phy.queensu.ca/~phil/exiftool/
MP3 - ID3 tag data
- Unknown. Search on http://search.cpan.org, update StreamCast with this information as well
libextractor should also handle the above filetypes

Version Numbering System

For each release, use the year then the release number for that year spelled out, using different languages. Each year, choose a different language, then start counting from one until whichever version ends up being the last version for that year

one, two, three, four
uno, dos, tres, cuatro

PortaboomHave you fragged someone today?

Main (edit sidebar)

Projects

AntLinux

PortaBoom

Streamcast

Streambake

Camelbox

Propaganda

Project Naranja

OSRPG

T.o.M.M

HOWTOs

Misc