|
|
|
When you deploy an instance of the WordHoard system, you must deploy six different parts of the system:
Each of these parts is largely independent of the other parts, and in fact each part may be deployed on a different server host.
The first three items, the static and dynamic databases and the server, are called a "WordHoard site". There may be more than one deployed site. The sites file enumerates all the deployed sites.
In this chapter, we use the "Russell" server at Northwestern University as an example. This is the host at russell.at.northwestern.edu. It is a Mac OS X system one of the NU developers uses to deploy development versions of software projects for local testing by colleagues and faculty partners. We use Russell to deploy all five parts of the WordHoard system for testing.
There are many pesky system administration details to keep track of when you create and maintain a deployed WordHoard system, so this chapter is quite long. We provide several shell scripts to help you generate the many files and directories needed for deployment and to perform other deployment tasks. All of these scripts are described in detail in the sections which follow. Some of them are run on your development host, and some of them are run on your deployment host or hosts.
All of the instructions and scripts described in this chapter are for deployment to a UNIX server or servers. While it is possible to deploy WordHoard on a Windows server, we do not provide any instructions or assistance for doing that.
The server to which you deploy the static database must be running a copy of MySQL version 5.0 or later.
To install the static database on the server for the first time, use the same steps you followed when you installed it on your development host for the first time. See the chapter Installing the Database for details. There are three main steps:
Step 1. Install the database files.
If you are installing the standard NU version of the static database, it is easiest to use the prebuilt zip archive that is part of our standard distribution. Download the archive directly to your server host, unzip it, and follow the instructions in the Installing the Database chapter to move the files into your MySQL data directory.
If you wish to install your own version of the static database, zip it up on your development host, transfer the archive to your server host, unzip it, and then follow the same instructions to move the files into your MySQL data directory. For example:
On your development host:
% mysql-stop % su Password: # cd $MYSQL_DATA # zip -r wordhoard wordhoard Use ftp, sftp, or whatever other method is convenient to copy wordhoard.zip to your server. # rm wordhoard.zip # <ctrl>d % mysql-start
Save the archive file wordhoard.zip on your server host. Don't delete it yet. You will need it later when you deploy the user manual.
Step 2. Create the grants.
The easiest way to create the MySQL grants on your server is to use the mysql-client-grants script. Copy the script file scripts/mysql-client-grants.csh from your development host to your server host, then run it on your server host. The script uses three environment variables that you must set to the proper values for your server's MySQL environment. For example:
On your server host, in the directory containing the script you copied:
% setenv MYSQL_ROOT_USERNAME root % setenv MYSQL_ROOT_PASSWORD my-password % setenv MYSQL_BIN /usr/local/mysql/bin % ./mysql-client-grants.csh
Step 3. Set MySQL configuration parameters.
Follow the instructions in the Installing the Database chapter to set the MySQL configuration parameters on your server.
You should also configure your server host to automatically start MySQL on server startup.
Step 4. Test the installation.
On your server host:
% mysql mysql> use wordhoard; mysql> show tables; +-----------------------------+ | Tables_in_wordhoard | +-----------------------------+ | annotation | | annotationcategory | ... etc. | workset | | workset_workparttags | +-----------------------------+ 48 rows in set (0.01 sec) mysql> show grants for wordhoard; +--------------------------------------------------------+ | Grants for wordhoard@% | +--------------------------------------------------------+ | GRANT USAGE ON *.* TO 'wordhoard'@'%' IDENTIFIED BY PASSWORD '*DD055982B03DB62CC130046066CE54FEE23EA734' | | GRANT SELECT ON `wordhoard`.* TO 'wordhoard'@'%' | +--------------------------------------------------------+ 16 rows in set (0.00 sec) mysql> show grants for wordhoardserver@localhost; +--------------------------------------------------------+ | Grants for wordhoardserver@localhost | +--------------------------------------------------------+ | GRANT USAGE ON *.* TO 'wordhoardserver'@'localhost' IDENTIFIED BY PASSWORD '*6A53706A4E45F99E396199F264AA4A606D29F6AB' | | GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, DROP, REFERENCES, INDEX, ALTER, CREATE TEMPORARY TABLES, LOCK TABLES ON `wordhoard`.* TO 'wordhoardserver'@'localhost' | +--------------------------------------------------------+ 2 rows in set (0.00 sec) mysql> select @@max_allowed_packet; +----------------------+ | @@max_allowed_packet | +----------------------+ | 16776192 | +----------------------+ 1 row in set (0.01 sec) mysql> quit
Installing a new version of the static database.
If you change any of the text or tagging data, or if you make any other changes to the structure or contents of the static database, you must deploy a new version of the database to your server.
This is a bit tricky because the static client database contains several tables for saved user data (e.g., saved work and word sets and user annotations). When you install a new version of the static database, you must not replace these tables, or you will destroy saved user data.
To help work around this problem, we have two scripts named save-user-tables and restore-user-tables. The first one saves the user tables in a directory named user-tables. The second one restores them from that directory and removes the directory. We recommend that you copy these two scripts to a convenient location on your server. The following example uses the scripts to deploy a new version of the static database.
On your server host, in the directory containing the two scripts and the unzipped "wordhoard" directory containing the new database files:
% mysql-stop % setenv MYSQL_DATA /usr/local/mysql/data % su Password: # ./save-user-tables.csh # rm -rf $MYSQL_DATA/wordhoard # mv wordhoard $MYSQL_DATA # chown -R mysql $MYSQL_DATA/wordhoard # chgrp -R wheel $MYSQL_DATA/wordhoard # ./restore-user-tables.csh # <ctrl>d % mysql-start
The server to which you deploy the dynamic database must be running a copy of MySQL version 5.0 or later.
For your convenience, we provide a prebuilt zip archive of the wordhoardserver database. Download this archive to your server host, unzip it, and use the usual technique to install it into the MySQL data directory. For example:
% mysql-stop % su Password: # setenv MYSQL_DATA /usr/local/mysql/data # mv wordhoardserver $MYSQL_DATA # chown -R mysql $MYSQL_DATA/wordhoardserver # chgrp -R wheel $MYSQL_DATA/wordhoardserver # <ctrl>d % mysql-start
Use the mysql-server-grants script to create the grants for the woardhoardserver database. Copy the script file scripts/mysql-server-grants.csh from your development host to your server host, then run it on your server host. For example:
On your server host, in the directory containing the script you copied:
% setenv MYSQL_ROOT_USERNAME root % setenv MYSQL_ROOT_PASSWORD my-password % setenv MYSQL_BIN /usr/local/mysql/bin % ./mysql-server-grants.csh
Test the installation:
% mysql mysql> use wordhoardserver; mysql> show tables; +---------------------------+ | Tables_in_wordhoardserver | +---------------------------+ | account | | userannotation | +---------------------------+ 2 rows in set (0.00 sec) mysql> show grants for wordhoardserver@localhost; +-----------------------------------------------------------------------+ | Grants for wordhoardserver@localhost | +-----------------------------------------------------------------------+ | GRANT USAGE ON *.* TO 'wordhoardserver'@'localhost' IDENTIFIED BY PASSWORD '*6A53706A4E45F99E396199F264AA4A606D29F6AB' | | GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, DROP, REFERENCES, INDEX, ALTER, CREATE TEMPORARY TABLES, LOCK TABLES ON `wordhoardserver`.* TO 'wordhoardserver'@'localhost' | +-----------------------------------------------------------------------+ 2 rows in set (0.00 sec) mysql> quit
You should configure your server host to automatically start MySQL on server startup.
The host you use to deploy the WordHoard server must be running the Java runtime environment 1.4 or later.
Before you use the following steps to deploy the server, you probably want to do a full build of the source code to make certain everything has been compiled correctly into your bin directory:
% b full
Step 1. Create the server directory.
Create a new directory named server in any convenient location on your server host. This directory will contain all the other WordHoard server files. Make the following three subdirectories:
lib. Contains all the third-party library jar files and the jar file for the WordHoard server proper.data. Contains the configuration and log files.bin. Contains scripts to start and stop the server.Step 2. Populate the lib directory.
To populate the lib directory, start by copying all of the third-party library jar files from the lib directory on your development host into the lib directory on your server host. Then create the wordhoardserver.jar file on your development host as follows:
% scripts/create-server-jar.csh
Copy wordhoardserver.jar from your development host into the lib directory on your server host.
Step 3. Populate the data directory.
The data directory contains the following files:
wordhoard.config. The WordHoard server configuration file.log.config. The server log configuration file.Start by creating the wordhoard.config file. This file contains configuration parameters for the WordHoard server. The standard settings are:
database-url jdbc:mysql://localhost/wordhoardserver? characterEncoding=UTF-8 database-username wordhoardserver database-password wordhoardserver userdata-database-url jdbc:mysql://localhost/wordhoard? characterEncoding=UTF-8 userdata-database-username wordhoardserver userdata-database-password wordhoardserver rmiregistry-port 1100 rmi-port 0
These settings will most likely work fine on your server, but you can change them if you want to. You can find a copy of this file in misc/server-wordhoard.config. If you wish, you can copy this file from your development host into the data directory on your server host, and rename it wordhoard.config. Note that in the example above we displayed the long lines for database-url and userdate-database-url split over two lines. This was done to make the example readable. There are not actually new lines in the file.
The WordHoard server uses both databases. The first group of three parameters database-url, database-username, and database-password are for the dynamic server database which contains the account objects. The second group of three parameters userdata-database-url, userdata-database-username, and userdata-database-password are for the static client database which contains saved user word sets, work sets, queries, and user annotations.
There are two port numbers for RMI.
rmiregistry-port is the port number used by the RMI registry. The WordHoard server creates and exports an RMI registry on the local host that accepts requests on this port number. The WordHoard client connects to this port number to get the server bootstrap object. Note that the WordHoard server creates its own private instance of an RMI registry running within its own process. It does not use a separate rmiregistry process that is shared with other applications. If you have other applications on your server host that use RMI, you may have to change the default port number 1100 to some unused port number for use by the WordHoard server to avoid conflicts with other applications that are using port 1100.
rmi-port is the port number used by the WordHoard server remote objects. A value of 0 for rmi-port tells the system to select an unused port number. If your server has a firewall which blocks ports, you will need to specify a fixed port number (e.g., 1101), and configure your firewall to permit traffic on that port number.
We use Log4J for server logging. The log.config file configures Log4J. We provide a default configuration in misc/server-log.config which rolls the log file every night at midnight. Copy this file from your development host into the data directory on your server host, and rename it log.config. You can use this sample as is, or make whatever changes you wish. See the Log4J documentation for details.
The WordHoard server writes its log files in the data directory. A message is written to the log every time someone runs the client, quits the client, logs in, or logs out. Any unexpected errors that happen in the client are also logged, with full stack traces. This is an indispensable debugging aid, since you do not have to rely on the users to send you accurate bug reports. We recommend checking the logs on a regular basis. On Russell we do this daily.
Step 4. Populate the bin directory.
The bin directory is a convenient location for helper scripts. We use a pair of scripts located in this directory to start and stop the server. Copies of these scripts are located in scripts/server-start.csh and scripts/server-stop.csh.
The server-start script starts the WordHoard server in the background:
#!/bin/csh cd /Users/wordhoard/server java -Xmx200m -jar lib/wordhoardserver.jar start data &
The server-stop script stops the server:
#!/bin/csh cd /Users/wordhoard/server java -jar lib/wordhoardserver.jar stop data
Note the path /Users/wordhoard/server in these scripts. This should be the path to your server directory on your server host. The sample files contain the proper path for our Russell server. The value on your system will be different, so you must make the appropriate changes to these two files.
You should configure your server host to automatically run the server-start script on server startup.
Step 5. Test the installation and start the server.
On your server host in your server directory:
% bin/server-start.csh WordHoard server started % cat data/log 2005-12-19 12:30:34,883 INFO - Server started, version 1.0fc2 % bin/server-stop.csh WordHoard server stopped % cat data/log 2005-12-19 12:30:34,883 INFO - Server started, version 1.0fc2 2005-12-19 12:30:36,116 INFO - Server stopped % bin/server-start.csh WordHoard server started
Leave the server running, assuming that you intend to finish the deployment shortly and go into production.
Step 6. Remember to change the admin password.
The empty wordhoardserver database you installed in step 1 contains no accounts (the account table is empty). The first time you ran the server in step 6, it created an initial administrator account with username "admin" and password "admin". As soon as you get everything deployed, you should use the WordHoard client to log in and change the password on this account.
Installing a new version of the server.
To install a new version of the server, first run the create-server-jar script on your development host to create the new version of the wordhoardserver.jar file:
% scripts/create-server-jar.csh
On the server host, change to your server directory and stop the WordHoard server:
% cd /Users/wordhoard/server % bin/server-stop.csh
Copy the new wordhoardserver.jar file from your development host to your server host, replacing the old copy in the server/lib directory, and then restart the server:
% bin/server-start.csh
The sites file, usually named sites.xml, enumerates all the known WordHoard sites. When a user runs the WordHoard client, it fetches this file from a web server. If there is more than one known site listed in the file, a site selection menu is presented.
For example, the following sites.xml file describes the example site we deployed to the russell server in the steps above.
<?xml version="1.0" encoding="utf-8"?> <WordHoardSites> <site id="main"> <name>Main Public Site</name> <description> This site contains the early Greek epics and the works of Shakespeare, Spenser and Chaucer. It is open to the world. </description> <database> <url>jdbc:mysql://russell.at.northwestern.edu/wordhoard?characterEncoding=UTF-8&useCompression=true</url> <username>wordhoard</username> <password>wordhoard</password> </database> <server>//russell.at.northwestern.edu:1100/WordHoard</server> </site> </WordHoardSites>
In this example, there is only one site listed. In this case, the WordHoard client connects directly to the site and does not present a site selection dialog.
Prepare your version of this file and deploy it to any convenient web server, typically the same web server that you are going to use to deploy the client and the user manual.
The is one site element for each WordHoard site. Each site must have a unique id.
The name and description elements are used to construct the site selection dialog.
The database element provides the connection information for the static database for the site: the JDBC URL, the username, and the password.
The server element provides the Java RMI URL of the WordHoard server for the site, without the leading schema name.
The file misc/sites.xml is the sites file that we use at Northwestern University for our WordHoard deployment.
In the next section we will discuss deploying the client for users to download via Web Start. Web Start requires that all of the jar files be digitally signed using a certificate.
For obvious reasons, we do not distribute our NU certificate in the public distribution of WordHoard for you to use to sign your modified versions of the program. You will have to use your own certificate.
If you are not familiar with Java's security tools, we recommend that you read Sun's descriptions of the Java keytool and jarsigner tools.
For developing and testing, you can use a self-signed certificate. Use keytool to create one. This tool manages a keystore (database) of private keys and their associated certificates. For example:
% mkdir private % keytool -genkey -keystore private/test-keystore -alias test Enter keystore password: testxx What is your first and last name? [Unknown]: Rocket J Squirrel What is the name of your organizational unit? [Unknown]: Department of Philosophy What is the name of your organization? [Unknown]: Wassamatta U What is the name of your City or Locality? [Unknown]: Frostbite Falls What is the name of your State or Province? [Unknown]: Minnesota What is the two-letter country code for this unit? [Unknown]: US Is CN=Rocket J Squirrel, OU=Department of Philosophy, O=Wassamatta U, L=Frostbite Falls, ST=Minnesota, C=US correct? [no]: yes Enter key password for <test> (RETURN if same as keystore password): %
In this example, our keystore is located at private/test-keystore. The keystore password is testxx. The keystore contains a single "alias" named test for our test certificate, also with password testxx.
For production use, you will want to obtain an official jar signing certificate from a certificate granting authority.
When a user downloads and runs WordHoard for the first time, Web Start presents a security alert that displays the information from your certificate. If you use a self-signed certificate, the alert contains dire language that recommends that the user not run the program. Using an official certificate results in an alert that is much less alarming, and reassures the user that they are obtaining the software from a reputable source.
The WordHoard client software can be distributed on any web server host. It is deployed on the web server as a directory named client which contains the following files:
wordhoard.jnlp. The Web Start deployment descriptor file. "jnlp" stands for "Java network launch protocol," which is the protocol used by Web Start.icon.gif. The WordHoard icon.wordhoard.jar. The main WordHoard jar file, which must be digitally signed using the certificate created in the previous section.You will do most of the work preparing your client deployment on your development system.
Step 1. Edit the wh.properties file.
Open the following source code file:
src/edu/northwestern/at/wordhoard/properties/wh.properties
Near the beginning of file you will find a section titled "Web site addresses":
# Web site addresses. programWebSite=http://russell.at.northwestern.edu/~wordhoard SitesURL=http://russell.at.northwestern.edu/~wordhoard/sites.xml
Edit these two web site addresses. programWebSite is the address of the user manual you will deploy later. SitesURL is the address of the sites file you deployed earlier.
Step 2. Do a full build.
Do a full build of the source code:
% b full
Step 3. Create the client directory.
Check your settings for the CODEBASE and HOMPAGE environment variables:
% echo $CODEBASE http://russell.at.northwestern.edu/~wordhoard/client/ % echo $HOMEPAGE http://russell.at.northwestern.edu/~wordhoard
These two variables specify the URL of your deployed client directory and user manual respectively. If they are not correct, edit your setup file and resource it (source setup). Then run the following script to create your client directory:
% scripts/create-client-dir.csh Enter store password for keystore: testxx Enter key password for alias test: testxx
This script creates your client directory. It creates the wordhoard.jnlp file in the client directory. It copies the icon file icon.gif from the misc directory into the client directory. It creates and signs the wordhoard.jar file. It copies all of the third-party library jar files from the lib directory into the client directory, then signs the copies using the certificate you created in the previous section.
Step 4. Check your web server settings.
Your web server must be configured to map the file name suffix jnlp to the MIME type application/x-java-jnlp-file. Most of the newer web servers we use these days are configured this way by default. Check to make certain, and add this mapping to your server's configuration if necessary.
Step 5. Deploy the client.
Create a directory named client in the proper location on your web server. Copy all of the files from the client directory on your development host to the client directory on your web server host.
Deploying a new version of the client
To deplay a new version of the client, create a new signed WordHoard jar file:
% scripts/create-client-jar.csh Enter store password for keystore: testxx Enter key password for alias test: testxx
Copy the new wordhoard.jar file to your web server, replacing the old copy of the file. The next time a user runs their copy of the client, Web Start will download your new jar file automatically.
The user manual (the one you are reading now) can be deployed to any web server. For example, on Russell, we organize the files and directories on the web server as follows:
% pwd /Users/wordhoard/Sites % ls client index.html sites.xml userman
The sites.xml file is the sites file described earlier.
The index.html file contains:
<html> <head> <meta http-equiv="Refresh" content="0; URL=http://russell.at.northwestern.edu/~wordhoard/userman/index.html"> </head> </html>
On Russell, a Mac OS X system running the Apache web server, the URL for our web site is:
http://russell.at.northwestern.edu/~wordhoard
This URL maps to the file /Users/wordhoard/Sites/index.html. This main index.html file in turn redirects the web browser to the URL:
http://russell.at.northwestern.edu/~wordhoard/userman/index.html
This URL is the table of contents page for this manual.
We recommend that you use the same web server for deploying both the user manual and the client, with the user manual directory named userman and the client directory named client siblings in the file system, as in the example from Russell above.
The reason the two directories should be siblings is because the user manual contains two references (links) to the wordhoard.jnlp file in the client directory. The two links are both titled "Download and Run WordHoard". One is located at the top of the table of contents page, and the other one is located in the middle of the Getting Started chapter. Both of these links are coded in the HTML source as follows:
<a href=../client/wordhoard.jnlp>Download and Run WordHoard</a>
If the userman and client directories are not siblings, these links will both break.
If you deploy the user manual and the client on different servers, or on the same server but not in sibling directories, you must take care to change the two links from the user manual to the client.
On your web server, the userman directory contains the HTML files for the manual, plus five subdirectories:
images. All the screen shots and other images for the manual.thirdparty. Third party licensing statements.javadoc. The full javadoc for WordHoard.data-files. Data zip archive files for developers.
data.zipwordhoard.zipwordhoardserver.zipother-files. Non-data zip archive files and other files for developers.On your development system, the HTML source for the user manual is located in the userman directory. This directory includes the images and thirdparty subdirectories, but not the other three subdirectories.
Step 1. Copy the userman directory to the web server.
Copy the entire userman directory from your development host to your web server.
Step 2. Create the javadoc subdirectory.
Build the javadoc on your development system:
% b doc
Copy the javadoc directory from your development host to your web server, and place the copy inside the userman directory.
Step 3. Create the other-files subdirectory.
% scripts/make-other-files.csh
This script makes the other-files directory. Copy it from your development host to your web server, and place the copy inside the userman directory.
Step 4. Create the data-files subdirectory.
On your web server, create a subdirectory named data-files inside your userman directory.
Move the archive file wordhoard.zip you saved when you installed the static database at the beginning of this chapter into the data-files subdirectory.
Run the following script on your development system to make a zip archive of the data directory:
% scripts/zip-raw-data.csh
Copy the data.zip file from your development host to your web server, and place the copy inside the data-files subdirectory.
On your web server, download a copy of the server database zip archive as the file named wordhoardserver.zip and place it inside the data-files subdirectory.
Test the installation.
Go to your new user manual web site address. Browse around in the manual to make certain all the chapters are intact, the screen shots are there, etc. Try downloading one of the developer zip archives. Click the "Download and Run WordHoard" link on the table of contents page. Verify that Web Start properly downloads the jar files and launches WordHoard. Log in using the "admin" account to verify that the server is working. This would be a good time to change the admin password.
Maintaining the user manual
You need to update your deployed user manual whenever you make changes to the HTML source or to the screen shots, of course.
Keep in mind that the user manual is also used to distribute files and the javadoc to developers. Whenever you deploy a new version of the system, you usually need to repeat steps 2 and 3 above to deploy the new javadoc and small files. If you made any changes to the raw data files or to the static database, you also need to repeat the appropriate parts of step 4.
As you do development work on WordHoard, you may find that on occasion you need to modify the collection of third-party libraries used by the system. You may add a new library, upgrade to a new release of an old library, or get rid of an old library. These libraries are all located in the lib directory on your development system.
During development, all you have to do is place the new libraries in your lib directory and source your setup script:
% source setup
The setup script redefines your classpath to include the new libraries.
In a WordHoard deployment, however, there are several places where you need to remember to make changes if you have modified the collection of third-party libraries in the lib directory. Use the following checklist to properly update the rest of the system at your next deployment:
server/lib. When you deploy, you must update this subdirectory to contain a copy of the new collection of library files.wordhoardserver.jar, which in turn contains a manifest that lists all of the third-party jar files. When you deploy, you must rebuild wordhoardserver.jar:
% scripts/create-server-jar.csh
The script scans your lib directory to rebuild the proper manifest.
Copy the new wordhoardserver.jar file from your development system to the server/lib subdirectory on your server.
client directory. When you add or update library files, you must make new signed copies for deployment. Use the sign-jar script to do this. For example, suppose you add a new library file named Goodies.jar to your collection, and you want to update the client directory so that it contains a signed copy of this jar file for deployment.
% scripts/sign-jar.csh Goodies.jar
Copy the new signed library file client/Goodies.jar from your development system to the client directory on your web server.
wordhoard.jnlp file for the deployed client contains a list of all the signed jar files. If you modify the collection of libraries, you must generate a new jnlp file:
% scripts/create-jnlp-file.csh
Copy the new jnlp file client/wordhoard.jnlp from your development system to the client directory on your web server.
For best results, avoid needlessly resigning old libraries or deploying new copies of the same jar files to web servers. If you do this, the next time a user runs WordHoard, he will have to wait while the "new" versions of the libraries are downloaded. Sign library jar files and deploy them only when they are actually new or changed. The same principle applies to generating and deploying new jnlp files. Do this only when necessary.
Note that we include all of the third-party library files in the deployments of both the client and the server, even though some of them are only needed for development, and of the remainder only subsets are needed by the client and server respectively. We could have and perhaps should have taken the trouble to modify our scripts to filter out libraries that are not needed in the different contexts. We have not bothered doing this.
|
|
|