DEL I: Prerequisites * Project plan Done 2002-10-22 doc/kravspec.txt (copied from meeting protocol) * Minimal prototype Done 2002-01-05 lib/usagechecker.py and lib/user.py * Name, logo and cvs-tree Done 2003-01-10 Navn: daddyQ CVS-tree: /home/stud/admin/cvs/daddyq (Now: cvs.daddyq.sourceforge.net/cvsroot/daddyq ) Logo: doc/img (Now: www/img) * Public webpage Done 2003-01-14 * Project plan (this document) Done 2003-01-10 doc/projectplan.txt * Project group members Stian, Dyre, Magnus, Sverre DEL II: Design and development We'll try to make an imprescion of doing Unified Process (UP) by UML (Unified modelling language - a way to draw circles and squares) and XP (Extreme Programming - we'll only use the two-person-programming principle) For a start, we'll only develop libraries for the most basic functions. Such libraries will be tested one at a time. We don't care of the database at this time, by using Stians forgetSQL-module we only need to design nice classes, and at any time replace them with instances from the databse. That means that we'll not define our sql tables at this time, but instead have example data (create example instances) * FileFinder Partly done - challenge: Doing several file servers * Example data - a module with instances of those classes that in the end will be stored in our database * FileChooser By Steinar Hamre bucket method. Note that we'll have to select files both for warning and for deletion, in the latter case we are not allowed to remove files that were not in the initial warning. We'll have to record which files are sent warning for, it is not decided if this is to be done in the database or in seperate file(s). * Deleter 'Deletes' files chosen by FileChooser by moving them to a different directory. Registers where the files have been moved, and when. (Actual deletion will be done by the Janitor) * TimeUsage Keeps tracks over how often a user has been over his quota the last N days. (say 30 days). If this number is over some limit, say 10 days, the warning sequence is started. * AccumulatedUsage Registers the grand total of overuse, degrading the existing grand total with some nice matemathical method like T=T*0.95. When over limit, the warning sequence is started. * Janitor Rotates 'deleted' files by keeping files for a maximum of 2 months and not more than 3 generations. That means a disk abuser with 3 deletions a week won't be able to restore files older than a week. (punishment) All program code will be written well documented. Generated API documentation should describe all modules, classes and functions. As a part of Unified Process we'll have to change classes and APIs on our way, so we need to keep track on dependencies and be aware of the patterns for low coupling and high cohesion. UML diagrams illustrating the code of today will be maintained on a weekly basis. Diagrams can be discussed on white boards, finding stupid things, fixing them, eventually resulting in a code rewrite. At this point it is important to let Sverre have his chocolate on request. DEL III: Database og final decission on class design * Our database will be designed at last, since it will be a partly object oriented database and should be a part of the UP design. At this date we should have decided on the class design for our data representation to be stored in the database (ie. Quota, Usage, File), so that desiging the database will be as simple as creating one table pr. class, recording attributes and linking between instances/tables. However, we should remember that things are going to be stored in the database from the beginning. * The database will be documented as one or several SQL files with CREATE and INSERT sentences. Connections between tables (ie. linking between class instances) MUST be documented as foreign keys in the SQL, even though our example database MySQL does not support foreign keys. SQL files must NOT be database dumps. There should be comments in the SQL code to explain difficult parts. * forgetSQL will be able to generate classes from the database, which we'll subclass to add our methods and attributes not to be stored in the database. DEL IV: Documentation * Documentation is in three parts: 1) For developers UML, generated API, documentation/comments in code, clean and structured code, database.sql 2) For system administrators Requirements for the system (Python version, database modules), installation, updating, configuration, which parts of the system does what, which servers will do what? How to fix things by 'going manual. Backup. How is logging done? What should be watched for in the logs? 3) For support and usres Example of site-specific routine for requests of extended quota and restore of deleted files. How are files selected for deletion? (simple explanation) How does punishment work, and why. DEL V: Testing and analysis We will always test subsystems on their own, but when time has come, we'll run our system in full scale, but in a passive mode. The system will check disk usage and register what actions would have been performed, but no warnings are sent, no files are deleted. Users will never know. On the basis of these data, a statistical analysis will be performed to check if our parameters chosen for accumulation degrade, limits for warnings, etc. are reasonable. We should be able to monitor that the file selector does a good job at this stage. =) We'll have to decide uppon storage of file lists for warning/deletion can be stored in the database or in seperate files. DEL VI: Presentation and administration * Disk usage and status stored in some XML form in users home directory, say ~/.diskusage.xml. The file will be owned by r00t. * This file will be read by a Visual Basic program written by Dyre. The script will be able to give immediate warnings to the user when he's over quota at login time (in Windows), and could be called seperately to check status. * A web presentation of usage, quota and grand total for a given user in a Customer centre web system, probably taken directly from the database. * A unix script «diskquota» to check the status of the user's disk usage at that time, and giving the same list of files «That could have been deleted». The user's actual quota cuold be read from ~/.diskusage.xml to avoid database connections from users. * A way to apply for extended quota on web. Existing code at NTNU might be reused. Work to be done on: 1) Webpages 2) Backend * A way to apply for restore of deleted files on web. Should check if actual disk usage is below limit before approval. Work to be done on: 1) Webpages 2) Backend DEL VII: Installation * Our old «period deletion» on NTNU is disengaged at the same time as daddyQ is enrolled all over. (ie. it is activated to actual perform warnings and deletions) * After a test period of say 1 month, a release is created and presented on the sourceforge site. DEL VIII: Paper and report * Stian and Dyre writes a nice paper on the whole subjects to be published in some fancy magazine. =)