Navigation

← Previous Changeset
Next Changeset →

Changeset 200

Timestamp:

08/24/08 10:05:11 (3 months ago)

Author:

rgrp

Message:

[shakespeare/misc][m]: fix up config registration, make associated changes to cli.py and move all tests to shakespeare/tests.

config setup:
- conf() in init was confusing, out of date, and was magically defaulting to 'development.ini'
- added new register_config and refactor conf to just look the config up
cli.py: refactored to take account of this adding new --config option to shakespeare-admin
tests: move all tests into shakespeare/tests so that config is properly set up for them

Files:

trunk/shakespeare.egg-info/paste_deploy_config.ini_tmpl (modified) (1 diff)
trunk/shakespeare/__init__.py (modified) (1 diff)
trunk/shakespeare/cli.py (modified) (1 diff)
trunk/shakespeare/index_test.py (deleted)
trunk/shakespeare/tests/test_cache.py (moved) (moved from trunk/shakespeare/cache_test.py)
trunk/shakespeare/tests/test_format.py (moved) (moved from trunk/shakespeare/format_test.py)
trunk/shakespeare/tests/test_gutenberg.py (moved) (moved from trunk/shakespeare/gutenberg_test.py)
trunk/shakespeare/tests/test_textutils.py (moved) (moved from trunk/shakespeare/textutils_test.py)

Legend:

: Unmodified
: Added
: Removed
: Modified
: Copied
: Moved

trunk/shakespeare.egg-info/paste_deploy_config.ini_tmpl

+#
 # shakespeare - Pylons configuration
+#
 # The %(here)s variable will be replaced with the parent directory of this file
+#
 [DEFAULT]
 debug = true
 email_to = you@yourdomain.com
 smtp_server = localhost
 error_email_from = paste@localhost
 # Cache directory where cached copies of downloaded materials can be stored
+#
 # This directory needs to be semi-permanent so do *not* put under a location
 # such as /tmp.
+#
 # At present should be different from the app's cache_dir
 cachedir = cache
 # Directory for Xapian search index
 search_index_dir = searchindex
 [server:main]
 use = egg:Paste#http
 host = 0.0.0.0
 port = 5000
 [app:main]
 use = egg:shakespeare
 full_stack = true
 cache_dir = %(here)s/data
 beaker.session.key = shakespeare
 beaker.session.secret = ${app_instance_secret}
 app_instance_uuid = ${app_instance_uuid}
 # If you'd like to fine-tune the individual locations of the cache data dirs
 # for the Cache data, or the Session saves, un-comment the desired settings
 # here:
 #beaker.cache.data_dir = %(here)s/data/cache
 #beaker.session.data_dir = %(here)s/data/sessions
 # WARNING: *THE LINE BELOW MUST BE UNCOMMENTED ON A PRODUCTION ENVIRONMENT*
 # Debug mode will enable the interactive debugging tool, allowing ANYONE to
 # execute malicious code after an exception is raised.
 set debug = false
-# using sqlite in memory leads to thread issues when using db ...
+# We use sqlalchemy to connect and work with databases.
-# sqlobject.dburi = sqlite:///:memory:
+# This default config using sqlite.
-sqlobject.dburi = postgres://<username>:<password>@localhost/<your-dbname>
+sqlalchemy.url = sqlite:///%(here)s/shkspr.db
+# This is for postgres (obviously change the values to those for your system)
+# sqlalchemy.url = postgres://<username>:<password>@localhost/<your-dbname>
 # Logging configuration
 [loggers]
 keys = root
 [handlers]
 keys = console
 [formatters]
 keys = generic
 [logger_root]
 level = INFO
 handlers = console
 [handler_console]
 class = StreamHandler
 args = (sys.stderr,)
 level = NOTSET
 formatter = generic
 [formatter_generic]
 format = %(asctime)s %(levelname)-5.5s [%(name)s] %(message)s
-[misc]
-# directory where we can store all local copies of texts
-cachedir = ./cache
-[db]
-# sqlobject database uri. see sqlobject documentation for details
-# uri = postgres://user:pass@host/dbname
-uri = sqlite:/:memory:
-[web]
-# directory where the templates used by web front end are kept
-template_dir = ./src/shakespeare/template
-[annotater]
-# url at which marginalia files (css/js etc) should be mounted
-marginalia_prefix = /marginalia

trunk/shakespeare/init.py

 '''
 Introduction
 ************
 The Open Shakespeare package provides a full open set of shakespeare's works
 (often in multiple versions) along with ancillary material, a variety of tools
 and a python API.
 Specifically in addition to the works themselves (often in multiple versions)
 there is an introduction, a chronology, explanatory notes, a concordance and
 search facilities.
 All material is open source/open knowledge so that anyone can use, redistribute
 and reuse these materials freely. For exact details of the license under which
 this package is made available please see COPYING.txt.
 Open Shakespeare has been developed under the aegis of the Open Knowledge
 Foundation (http://www.okfn.org/).
 Contact the Project
 *******************
 Please mail info@okfn.org or join the okfn-discuss mailing list:
   http://lists.okfn.org/listinfo/okfn-discuss
 Installation and Setup
 **********************
 . Install the code
 ===================
 .1: (EITHER) Install using setup.py (preferred)
 ------------------------------------------------
 Install ``shakespeare`` using easy_install::
     easy_install shakespeare
 NB: If you don't have easy_install you can get from here:
 <http://peak.telecommunity.com/DevCenter/EasyInstall#installation-instructions>
 .2 (OR) Get the code straight from subversion
 ------------------------------------------------
 . Check out the subversion trunk::
     svn co https://knowledgeforge.net/shakespeare/svn/trunk
 . Do::
     sudo python setup.py develop
 Getting Started
 ***************
 As a user:
 ==========
 . Basic setup
 --------------
 To access most of the main features of Open Shakespeare you need a database.
 For this an other bits and bobs of configuration you will need a configuration
 file.
 You can make a config file as follows::
     paster make-config shakespeare {your-config.ini}
 Tweak the config file as appropriate and then setup the application::
     paster setup-app config.ini
 [TODO: this should be part of setup-app]
 Run::
-    $ shakespeare-admin db create
     $ shakespeare-admin db init
 . Extras
 ---------
 . Search index. [TODO]
 . You can start a web server to provide a easy-to-use web interface to the
 shakespeare material and facilities by doing::
     $ paster serve {your-config.ini}
 NB: {your-config.ini} should be replaced with the name of the config file you
 created earlier.
 As a developer:
 ===============
 . Setup
 --------
 Follow the basic steps above put with an ini file named: development.ini
 NB: you'll probably want to change log levels to debug.
 . Check out the administrative commands
 ----------------------------------------
     $ bin/shakespeare-admin help.
 . Run the tests using either py.test of nosetests::
 ----------------------------------------------------
     $ nosetests shakespeare
 '''
 __version__ = '0.6a'
 __application_name__ = 'shakespeare'
-def conf():
+def register_config(config_path):
     import os
-    defaultPath = os.path.abspath('./development.ini')
+    # TODO: remove? 2008-08-24 not mentioned in docs any more
-    envVarName = __application_name__.upper() + 'CONF'
+    # envVarName = __application_name__.upper() + 'CONF'
-    confPath = os.environ.get(envVarName, defaultPath)
+    # config_path = os.environ.get(envVarName, '')
-    if not os.path.exists(confPath):
+    config_path = os.path.abspath(config_path)
+        raise ValueError('No Configuration file exists at: %s' % confPath)
+    # register the config
     import paste.deploy
+    pasteconf = paste.deploy.appconfig('config:' + config_path)
     import shakespeare.config.environment
-    pasteconf = paste.deploy.appconfig('config:' + confPath)
     shakespeare.config.environment.load_environment(pasteconf.global_conf,
         pasteconf.local_conf)
+# TODO: rename to get_config()
+def conf():
     from pylons import config
     conf = config
-    # import ConfigParser
-    # conf = ConfigParser.SafeConfigParser()
-    # conf.read(confPath)
     return conf

trunk/shakespeare/cli.py

 #!/usr/bin/env python
 import cmd
 import os
 import StringIO
 class ShakespeareAdmin(cmd.Cmd):
     """
     TODO: self.verbose option and associated self._print
     """
-    def __init__(self, verbose=False):
+    def __init__(self, config=None, verbose=False):
         # cmd.Cmd is not a new style class
         cmd.Cmd.__init__(self)
+        self.config = config
         self.verbose = verbose
     def _print(self, msg, force=False):
         if self.verbose or force:
             print msg
-    prompt = 'The Bard > '
+    def _register_config(self):
+        import sys
-    def run_interactive(self, line=None):
+        if not self.config:
-        """Run an interactive session.
+            msg = 'No configuration file has been specified. See -h help for details'
-        """
+            print msg
-        print 'Welcome to shakespeare-admin interactive mode\n'
+            sys.exit(1)
-        self.do_about()
+        import shakespeare
-        print 'Type:  "?" or "help" for help on commands.\n'
+        shakespeare.register_config(self.config)
+        while 1:
+            try:
+                self.cmdloop()
+                break
+            except KeyboardInterrupt:
+                raise
     def do_help(self, line=None):
         cmd.Cmd.do_help(self, line)
     def do_about(self, line=None):
         import shakespeare
         version = shakespeare.__version__
         about = \
 '''Open Shakespeare version %s. Copyright the Open Knowledge Foundation.
 Open Shakespeare is open-knowledge and open-source. See COPYING for details.
 For more information about the package run `info`.
 ''' % version
         print about
     def do_quit(self, line=None):
         sys.exit()
     def do_EOF(self, *args):
         print ''
         sys.exit()
     # =================
     # Commands
     def do_db(self, line=None):
-        actions = [ 'create', 'clean', 'rebuild', 'init' ]
+        actions = [ 'create', 'clean', 'init' ]
         if line is None or line not in actions:
             self.help_db()
             return 1
+        self._register_config()
         import shakespeare.model
+        import shakespeare
         if line == 'init':
             import pkg_resources
             pkg = 'shksprdata'
             meta = pkg_resources.resource_stream(pkg, 'texts/metadata.txt')
             shakespeare.model.Material.load_from_metadata(meta)
-        else:
+        elif line == 'clean':
+            config = shakespeare.conf()
+            shakespeare.model.metadata.drop_all(bind=config['pylons.g'].sa_engine)
+        elif line == 'create':
             print 'To create db use paster: paster setup-app {config-file}'
+        else:
+            print self.help_db()
     def help_db(self, line=None):
         usage = \
 '''db { create | init }
 '''
         print usage
     def do_gutenberg(self, line=None):
+        self._register_config()
         import shakespeare.gutenberg
         helper = shakespeare.gutenberg.Helper(verbose=True)
         if not line:
             helper.execute()
         elif line == 'print_index':
             import pprint
             pprint.pprint(helper.get_index())
         else:
             msg = 'Unknown argument %s' % line
             raise Exception(msg)
     def help_gutenberg(self, line=None):
         usage = \
 """
 Download and process all Project Gutenberg shakespeare texts"""
         print usage
     def do_moby(self, line=None):
         import shakespeare.moby
         helper = shakespeare.moby.Helper(verbose=True)
         if not line:
             helper.execute()
         elif line == 'print_index':
             import pprint
             pprint.pprint(helper.get_index())
         else:
             msg = 'Unknown argument %s' % line
             raise Exception(msg)
     def help_moby(self, line=None):
+        self._register_config()
         usage = \
 '''
 Download and process all Moby/Bosak shakespeare texts'''
         print usage
     def _init_index(self):
+        self._register_config()
         import shakespeare.index
         self._index = shakespeare.index.all
     def _filter_index(self, line):
         """Filter items in index return only those whose id (url) is in line
         If line is empty or None return all items
         """
         if line:
             textsToAdd = []
             textNames = line.split()
             for item in self._index:
                 if item.name in textNames:
                     textsToAdd.append(item)
             return textsToAdd
         else:
             self._init_index()
             return self._index
     def do_index(self, line):
         self._init_index()
         header = \
 '''          +-------------------+
           | Index of Material |
           +-------------------+
 '''
         print header
         for row in self._index:
             print row.name.ljust(35), row.title
     def help_index(self, line=None):
         usage = \
 '''Print index of Shakespeare texts to stdout'''
         print usage
     def do_runserver(self, line=None):
         self.help_runserver()
     def help_runserver(self, line=None):
         usage = \
 '''This command has been DEPRECATED.
 Please use `paster serve` to run a server now, e.g.::
     paster serve <my-config.ini>
 '''
         print usage
     def do_info(self, line=None):
         import shakespeare
         info = shakespeare.__doc__
         print
         print '       ## Open Shakespeare ##'
         print info
     def help_info(self, line=None):
         print 'Information about this package.'
     def _parse_line(self, line):
         line = line.strip()
         args = line.split()
         action = ''
         remainder = ''
         if len(args) > 0:
             action = args[0]
         if len(args) > 1:
             remainder = ' '.join(args[1:])
         return (action, remainder)
     def do_search(self, line):
+        self._register_config()
         import shakespeare.search
         index = shakespeare.search.SearchIndex.default_index()
         action, extra = self._parse_line(line)
         if action == 'addpath':
             index.add_from_path(extra)
         elif action == 'query':
             results = index.search(extra)
             print index.print_matches(results)
         elif action == 'addtext':
             import shakespeare.model as model
             text = model.Material.byName(extra)
             fileobj = text.get_text()
             index.add_item(fileobj)
         elif action == 'init':
             self._init_index()
             for text in self._index:
                 # exclude folios as many odd spellings
                 if text.name.endswith('_f'):
                     continue
                 self._print('Adding: %s' % text.name)
                 fileobj = text.get_text()
                 index.add_item(fileobj)
         else:
             print 'Unrecognized action: %s' % action
             self.help_search()
             return 1
     def help_search(self, line=None):
         info = \
 '''
 search addpath {path}
     - Add contents of {path} (file itself or all text files in directory if
       directory) to the search index.
 search addtext {name}
     - Add db text named {name} to search index.
 search query {query}
     - Query search index with {query}.
 search init
     - Add all texts in DB to index.
 '''
         print info
     def do_stats(self, line):
+        self._register_config()
         action, extra = self._parse_line(line)
         import shakespeare.stats
         stats = shakespeare.stats.Stats()
         if action == 'init':
             self._init_index()
             for text in self._index:
                 # exclude folios as many odd spellings
                 if text.name.endswith('_f'):
                     continue
                 self._print('Adding: %s' % text.name)
                 stats.statsify(text, text.get_text())
         elif action == 'addtext':
             import shakespeare.model as model
             text = model.Material.byName(extra)
             stats.statsify(text, text.get_text())
         elif action == 'show':
             textstats = stats.text_stats(extra)
             for s in textstats:
                 print s.word, s.freq
         else:
             print 'Unrecognized action: %s' % action
             self.help_stats()
             return 1
     def help_stats(self, line=None):
         info = \
 '''
 stats addtext {name}
     - Add db text named {name} to stats index.
 stats show {name}
     - Query stats index with {query}.
 stats init
     - Prepare statistics for all texts in DB.
 '''
         print info
 def main():
     import optparse
     usage = \
 '''%prog [options] <command>
-Run about or help for details.'''
+For list of the commands available run:
+    $ shakespeare-admin help
+For more general information run the about or info commands.'''
     parser = optparse.OptionParser(usage)
     parser.add_option('-v', '--verbose', dest='verbose', help='Be verbose',
             action='store_true', default=False)
+    parser.add_option('-c', '--config', dest='config',
+        help='Path to config file', default=None)
     options, args = parser.parse_args()
     if len(args) == 0:
         parser.print_help()
         return 1
     else:
-        cmd = ShakespeareAdmin(verbose=options.verbose)
+        cmd = ShakespeareAdmin(verbose=options.verbose, config=options.config)
         args = ' '.join(args)
         args = args.replace('-','_')
         cmd.onecmd(args)