Hide Notes 

Slide 1: Codendi Administration Guide Copyright © Xerox Corporation, Codendi 2007-2009. This file is licensed under the GNU General Public License version 2. See the file COPYING. Table of Content Administration Tasks Codendi File Permissions Codendi File Structure Codendi Databases Apache FTP CVS DNS SSH Sendmail Mailman OpenFire Background Daemons Log Files Codendi Code Update Classic update in console environment Web interface update mechanism Backup/Restore Codendi Testing Environment Codendi HowTo Importing an existing CVS repository in Codendi Importing an existing Subversion repository in Codendi Regenerating a fresh CVS or SVN repository for a Codendi project Changing a User Name on Codendi Changing a Project Short Name on Codendi New user registration problems - email address errors Blocking a Client from Accessing Codendi How to suspend/delete a Codendi user Recompiling a Codendi specific RPM Change privileged account passwords Delete a bug from the Codendi database Enabling 'Restricted Users' on a Codendi server Localize service names Convert a CVS repository to Subversion Restore a Subversion repository Recover a Subversion repository Clean-up a Subversion repository
Slide 2: Set up a distributed Architecture Set up a validator for users' password How to add an expiration date on a user account How to change wiki language for a project Administration Tasks To access the Codendi Administration interface you must use an account with administration privileges. When Codendi is first installed a privileged account named 'admin' is created with the default password 'siteadmin'. One of the first thing you must do is to login as 'admin' and change the password for security reasons. Note that any normal user accounts can also be granted site administration privileges by her a member of the project id #1, the Codendi Administration project, and giving her the status of administrator on this project. ** IMPORTANT REMARK ** When you use a Codendi account with administration privileges, remember to log out after each work session especially if you work an a desktop computer other than yours or if you share a computer with other people Pending projects Before a Codendi hosted project is made available to the project team, it must be reviewed and approved by the Codendi administration team. New projects must be checked for approval on a regular basis so that the submitter doesn't wait too long before they get the confirmation. When a project is submitted an email notification is sent to the Codendi administrators (see the $sys_email_admin variable in the Codendi configuration file) allowing for a quick response time. In general it is recommended that you approve all projects. There most noticeable exceptions are: 1. Little pieces of code If the submitted project looks like it is going to post small pieces of code like a macro or a function or a simple script, the administrator reviewing the project submission must contact the project creator and suggest that the Codendi Snippet Library might be a better place to host such a small piece of code. 2. Yet Another Test Project The second case is when the team is saying they are creating a file for testing purposes. In order to avoid the proliferation of test projects some generic purpose test projects have been created. In this case the project creator must be contacted and the admin must suggest that we make him/her a member of the Guinea Pig test project with full admin rights so that s/he can test all the Codendi feature and make a conscious decision. If it is ok then we do approve the project. 3. Similar projects exist As a site administrator, you'll have the opportunity to review all the projects created on Codendi and, therefore, you have a unique opportunity to advise project leaders about existing projects looking like the one you are about to approve. This might be a chance for the project submitter to learn about another Codendi project that does more or less what s/he wants and, rather than created a new project, it may be more appropriate to join the existing team and contribute to the other project.
Slide 3: Pending users Before a user account is made available, it has to be activated. According to your Codendi configuration there are several ways to activate an account: • • • Standard case, no administration approval is needed. Once the user created his account, he will receive an e-mail, and then when clicking on the link inside he will activate his account. The admin can also activate the account (if the user e-mail is not valid...) in the pending user interface. Administration approval is needed (sys_user_approval = 1). Once the user created his account, the administrator is notified by e-mail. He has to first validate the account in the pending user interface. Then an email is sent to the user. The account is activated once the user has followed the link sent in the email. The administrator can also directly activate the account : • after validation, in the validated user interface • or before validation directly in the pending user interface Restricted user and administration approval is needed (sys_allow_register_user=1 and sys_user_approval=1) Once the user created his account, the administrator is notified by e-mail. He has to validate the account in the pending user interface. He can choose between standard user or restricted user. Then an email is sent to the user. The account is activated once the user has followed the link sent in the email. The administrator can also directly activate the account : • after validation, in the validated user interface • or before validation directly in the pending user interface See also Enabling 'Restricted Users' on a Codendi server Here are the various possible status values for a Codendi user: • • • • • • • Pending (internal: 'P'): the account was submitted, but not validated or activated. Validated (internal: 'V'): (Only when administrator approval is needed): an administrator validated the account. It is not active yet: the user has to follow the link sent in the validation email to activate the account. The administrator may also activate the account in the administration interface. Validated as Restricted (internal:W): This is the same as above: the account was validated by an admin, but as 'restricted'. Active (internal: A): The account is active: the user can log in and use Codendi. Restricted (internal: R): The account is active with a restricted status. The user can log in and use Codendi, but only on projects he belongs to. Suspended (internal: S): The account was suspended by an administrator: the user can no longer log in. Deleted (internal: D): the account was deleted and the user home directory on Codendi is removed, as well as all entries in system files. Site News Approval All news published by each project (using the News service) is normally only visible on the related Project Summary page. The Codendi Administration team has the ability to review all pieces of news published over the last 7 days by the various projects and make the decision to promote any piece of news so that it appears on the Codendi front page. If you decide a news is worth a big splash on the front page then select "Approve for front page", make modifications in the news title or content to make it more explicit and validate. If not interesting then you can delete the news front the approval queue (it is not deleted from the project news
Slide 4: database of course!) Rules to follow to determine whether a piece of news is worth publishing on the front page: • • • All news dealing with new software releases and other significant events in the life of the projects (new members,...) should be promoted on the front page The same applies for news announcing the creation of a new project. However the Codendi front page already has a section automatically showing the most recent projects and you may decide not to publish new project announcement. If a project publishes 2 or 3 pieces of news at the same time, then select the most significant one and only publish this one. When the Codendi visitors click on this particular piece of news they will be redirected to the project pages and they get a chance to read the other news. User Accounts Statuses There are 2 different statuses associated with a Codendi user: 1. The "status" field in the user table contains the user status from the point of view of the Web interface. • Valid values are : • 'A' for Active • 'P' for Pending • 'V' for Validated • 'R' for Restricted • 'W' for Validated as Restricted • 'S' for Suspended • 'D' for Deleted • 'Validated':(Only when administrator approval is needed) an administrator validated the account. It is not active yet: the user has to follow the link sent in the validation email to activate the account. The administrator may also activate the account in the administration interface. • 'Restricted' users are like active users with restricted access rights:they cannot access pages from projects they are not member of. This special role is only enabled on Codendi servers in heterogeneous environments where corporate users and external consultants share the same server. • 'Validated as Restricted' is the same as 'Validated', the account was validated by an admin, but as 'restricted'. • From the point of view of the Web interface there is not much difference between 'S' and 'D': in both cases the effect is that the user can non longer login and in both cases they are not counted in the number of active users (upper right corner of the front page). In any case a user will never disappear from the MySQL database. • To modify this field one has to go either to the User Admin page (left menu pane) but it generates a huge page with all users. So a better choice is to choose "Admin Page" and then search for the use or select the first letter of the name. Then you can change the Web status. 2. The "unix_status" field in the user table governs the status of the Unix account. • It is completely independent from the Web 'status' field above • Valid values are • 'N' for No Unix account: this one has an effect only when a user is created with this unix status upfront. In this case the Unix crontab daemon simply ignores it and doesn't create a Unix account for this user • 'A' for Active: A Unix account is created for this user (including a home directory
Slide 5: • in /home/users/user_name) • 'R' for Restricted: A Unix account is created for this user (including a home directory in /home/users/user_name) • 'S' for Suspended: the Unix password is replaced by "!!" meaning the user account is preserved although no longer usable (Can not login but home directory remains untouched and assignment to Unix group is safe as well).Going back to status 'A' will reactivate the account with the initial password. • 'D' for Deleted: the home directory for this user is archived in /tmp (and therefore automatically cleaned up after 7 days) for the moment and then removed from /home/users. Assignment to Unix groups is revoked as well of course. To modify the Unix status do the same as for status but click on the user name. Then you are given access to the Unix status of the user. Change it to whatever value is appropriate. Account Management Codendi is a self managed system and there is normally no need to spend time on user account maintenance. There are a couple of exceptions though: 1. Lost password: some user sometimes ask for their lost password. Direct them to the Codendi login screen and ask them to click on the 'Lost Password' link. 2. Lost login: some users even lose their login name :-) Tell them to use the Codendi search box, select the People item and type a part of their last name (it's very unlikely that they also forgot about their last name...). The login should appear in the result list and they can then follow the lost password procedure as explained before. 3. Lost password and e-mail address has changed: in this case the normal recovery procedure won't work because the user will never receive the e-mail notification (given that his/her email address is wrong). The Codendi Administrator must use the User Administration module, update the e-mail address of the user and then tell the user to proceed as in the lost password procedure. 4. E-mail address is no longer valid: this is something that is often seen when using the Codendi mass mail Engine. All invalid e-mail address are bounced and returned to codendiadmin@codendi.example.com. Once in a while it is good to make a bit of clean-up in the user database as follows: • Look for the person name in your enterprise directory. If this person is still working for the company and his/her e-mail address has changed then contact the user and ask him/her to update the address. • If the person is no longer with your company, go to the Codendi User Administration page, spot the user login name and click either on 'Suspend' or 'Delete' link for this account. Opting for account suspension gives a chance to the user to complain that she is still alive and we can easily reactivate the account without loosing personal information. 5. Incorrect e-mail given at registration time: amazingly enough this is quite a common mistake for new user to mistype their email address on the user registration form. In this case, the email notification sent to the user for confirmation never reaches the recipient and the user account cannot be validated . Upon reception of the bounced email notification, the Codendi Administrators have two options: • access the Codendi main administration page, look for pending user account, force the approval of the user account and send a note to the user saying that the account is now active and the first she must do is to change her email address under the 'Account Maintenance' link • or the Codendi Administrator can simply send the bounced message to the appropriate email address, ask the user to confirm his registration and change her email address as above. 6. Alien e-mail addresses: when a user registers we strongly recommend that they use internal and approved e-mail address only. Ideally the user should also use the canonical form of her email address
Slide 6: type it is it appears in your company directory. By doing so, Codendi look-ups in your company directory will work correctly. 7. Create a new user account: in the Admin interface by clicking on the 'new user' link in the user administration part. The interface is nearly the same as the one of account creation by a user. You have to choose the login and password of the new user. You can choose to create the user as a restricted user by selecting the box 'Restricted user' on the bottom of the page. By default no email is sent to the new user, so please remember the login and password you set for the user. If you want that the user receives a welcome email with the login and password from Codendi, check the box 'Send a welcome email to the user' before activating the account. Site News Push Codendi offers a mass mail engine to the Codendi Administration Team making it possible to push an announcement to a all or part of your Codendi population. Be careful that it's ideal for spamming so use it with caution and only when needed (see below). The mass mail engine allows you to select your target population, type a message and send it with a click of a mouse. Use this mechanism to push site update news like: • • • • A new Codendi document is published: Codendi newsletter, Codendi article, new Codendi User Guide,... New major features available in Codendi Maintenance operation: hardware or software upgrade and all the events that will prevent the Codendi server from operating normally must be announced ahead of time Disaster reports: typical examples are network problems due to router flapping or wide area network cut due. Codendi has nothing to do with this kind of troubles but we must inform the user that they'll probably experience some difficulties to reach the Codendi server Remark: when a mass mail message is sent the Codendi Administration team may receive many copies of it. This is due to the fact that messages are sent by chunk of 25 addresses and Codendi administrators receive a copy for each chunk. So the larger the selected population the bigger the number of copies. The reason for these multiple copies is because messages are "apparently" sent to noreply@codendi.example.com to stress the fact that recipient should not reply to this message. If the noreply email address is an alias to codendiadmin then multiple copies will be received by Codendi administrators. The noreply address may also be aliased to /dev/null to avoid this problem (see /etc/aliases). Developer Survey The Developer Survey is a special survey that appears in the user's personal page. The developer is a general survey that gives site administrators the opportunity to better understand the population of users (e.g. how they heard about Codendi, do they like it, etc.). This survey can only be answered once. The Developer Survey is automatically created at Codendi installation time (survey id 1, associated to project 1), but it is inactive by default and does not contain any question. Codendi File Permissions The Codendi Web interface use the paradigm of project and project members to manage access permissions
Slide 7: to the various project data. Codendi maintains this access permission policy both at the Web and Unix levels. Therefore Codendi makes a direct mapping between a project and a Linux group, and between a user and a Linux account. As an example if you are a member of the "codendi" project on Codendi, your Unix account will also be a member of the "codendi" Unix group. Let's review some of the key permission mechanism in place on Codendi. More information on the subject can be found in the Codendi User Guide User accounts When a new user registers on Codendi three accounts are created: 1. A Web account managed by the login/session mechanism implemented in PHP: Information about login name and encrypted password are stored in the Codendi database. 2. A Unix account with the same name/password as the Web account This account is used for shell account and to setup the ownership on appropriate directories. Information about login name and encrypted password are stored both in the Codendi database user table and in the Linux password and group files. User IDs (uid) are in the range (20000-49999), the uid number is determined when the user account is first created on the Codendi Web front-end. It's a sequential number incremented for each newly created account. Account ID of deleted users are not re-used. Remark: Unix login name are all lowercase even if uppercase are used in the Web login name. There are specific user account you must know about: • • • • codendiadm: this is the user that owns the entire Codendi Web site hierarchy under /usr/share/codendi dummy: a "dummy" user that is generally used whenever a file belongs to a given group but to no specific user in particular. More or less equivalent to the "nobody" user. mailman: owner of the mailman directories anoncvs_xxxx (deprecated - login no longer created): a specific user created for each project where xxxx is the project short name . These are not real login account but are just used to give source code checkout capabilities to anonymous users. By default Codendi does not allow anonymous CVS access and these accounts are not used unless a project explicitly asks for anonymous CVS access to be activated.We keep maintaining them in case a project would like to activate anonymous read access on their CVS repository. uid range is 50000-65535. Group accounts When a Codendi hosted project is created the project administrator gives it a project short name. This project short name can be used in a number of places: 1. http://codendi.example.com/projects/ProjectShortName is a shortcut to access a project summary page on the Codendi Web site (not to be confused with the Project Web site at http://ProjectShortName.codendi.example.com/ 2. A Unix group with the same name (all lowercase) is created
Slide 8: • • It is applied to all projects related disk resources (e.g. the Project home directory where the project web site is hosted). Members of this group are all the developers who are declared as "project members" by the project administrator in the Project Admin page. In other word there is a direct mapping between project membership as defined through the Web front-end and group membership on the Unix side. 3. A Unix group with the same name as the user login name is also created for each user. So by default each user is a member of a group of his own. These groups are used for user home directories for instance so that only the user can access its own home directory. There are specific group accounts you must know about: • • codendiadm: group created to own the Codendi Web hierarchy under /usr/share/codendi. mailman: group owner of all the Mailman directories Codendi File Structure Web site The Codendi Web front end files are all under /usr/share/codendi. Except otherwise specified this directories and all the files and directories under it belongs to user.group: codendiadm.codendiadm (rwxrxr-x) Location src src /www Description The directory where all PHP scripts of the Web front end as well as back-end scripts reside. This is where all the PHP scripts are. Most of them ends with a .php extension except for a few o them that are used as shortcuts to projects and users information: • • projects/xxxx: goes to Summary page of project xxxx users/yyyy: goes to Profile page of users yyyy src/www/inc There are many PHP files in there used by all other scripts. They are sort of small modules lude providing user/login/session management, HTML formatting, DB connection. pre.php is the top include file used by all others. src/utils src/db This is where all the back-end Perl scripts are located. Contains the SQL batch files to create and populate the Codendi database. it also contains all the database patch file published by Xerox to correct Codendi defects or add new features. (see Codendi databases section for more information). src/back-end This subdirectory is not used by Codendi because all the configuration files contained in this
Slide 9: directory are elsewhere on the file system. See the src/backend/zones/README for more information src/etc This is the directory where a number of Codendi configuration files are located. These template files (hence the .dist extension) are instantiated by the Codendi installation script, customized at installation time and copied over to the target directories. Remark: the database.inc configuration file stored in /etc/codendi/conf/database.inc contains the Codendi database password in clear so it must be readable only for the codendiadm user/group only (rw-r-----) /usr/lib/code There are a couple of executables in this directory that work in cooperation with the Codendi ndi/bin WEB Site: • • • • fileforge: when a new file release is created and validated, attached uploaded files are moved into the File release space (runs suid root because it must read/write in all project space areas) gotohell: a small shell script used to redirect some emails (e.g noreply address) to a black hole! log_accum and commit_prep: a set of Perl scripts that send an email notification on a CVS commit. These script are used by Codendi whenever a project activate the email notification in the CVS Administration interface of Codendi. commit_email.pl: a Perl script used that send an email notification on a Subversion commit. This script is used by Codendi whenever a project activates the email notification in the Subversion Administration interface of Codendi. CVS Repositories Location Description /var/lib/codendi The CVS repositories of all Codendi projects are created under the /cvsroot and /var/lib/codendi/cvsroot directory. This directory is linked from /cvsroot in link the top directory. /var/lib/codendi/ProjectName is the CVS repository of project from /cvsroot ProjectName. By default all files belong to user nobody and group ProjectName. File permission is read only (r--r--r--) for files, while directories are user and group writable (rwxrwxr-x). This is defined by CVS and must not be touched. By default CVS servers on Codendi are configured to perform all transactions through the pserver protocol. Codendi can be tuned to use the ssh tunnelling protocol, or the sserver encrypted protocol (with CVSNT) instead. Ask the Xerox Codendi team for more information. If secured transaction and fine grain access permissions are high on your list of project requirements, we strongly advise you to consider the use of Subversion instead of
Slide 10: CVS. Exceptions to these rules are: • • • • CVS files that correspond to executables for which we want the executable flags to be preserved when the file is checked out of the CVS repository must have the executable flag set (GNU CVS only) The CVS lock dir is created in /var/lock/cvs/ProjectName. It is world writable (drwxrwxrwx) so that all users even those not part of the project team can checkout a working copy of the CVS tree. The CVS repository for each project is owned by the user codendiadm. This allows the CVS web interface, which implements its own access control, to access the directory even if the directory is not world readable. For private projects and public projects that set the CVS reposiotry to be private, the CVS repository is NOT world readable (drwxrwx---). It is only user and group readable because the sources must be invisible to non project members. Ability to commit changes to the CVS repository from a working copy is controlled by the CVSROOT/readers and CVSROOT/writers files. Those users in the writers file (all project members as of today) can commit changes to the CVS repository. Project administrators who wants to prevent some project members to commit changes in the project CVS repository should add the corresponding login names in the readers file (the writers file must not be touched). Subversion Repositories Location Description /var/lib/coden The Subversion repositories of all Codendi projects are created under the di/svnroot /var/lib/codendi/svnroot directory. This directory is linked from /svnroot in the and link from top directory. /var/lib/codendi/svnsroot/ProjectName is the Subversion repository of project /svnroot ProjectName. As opposed to CVS, Subversion manages the repository in a database acting as a virtual filesystem. Therefore access permission do not rely on the Linux file system permission but are directly defined and managed through the Subversion database. Project members and non project members permission can be defined through the Web interface of the Codendi Subversion service (see the Codendi User Guide for more details). Beside the virtual file system managed in the Subversion databases there are a number of files installed on the Linux file system that you must know about: • /svnroot/ProjectName/.SVNAccessFile: this file holds the definition of the access permission for a given Subversion repository. We highly recommend that you do not modify this file by hand but rather use the Web interface of the Codendi Subversion service to do so.
Slide 11: • /svnroot/ProjectName/hooks/post-commit: this hook file is installed by Codendi in order to feed the Codendi database with a number of information about all your Subversion transactions. This is the so called Subversion tracking feature that a project administrator can active or deactivate from the Web interface of the Codendi Subversion service. This file must not be modified or removed. Other hook scripts can be used by the project team if they have a shell account available on the server. Log files Location Description /var/log/code The directory where log files used for statistics are archived. Archiving takes place each day. ndi ./cvslogs/Yea File name pattern: cvs_traffic_YYYYMMDD.log r/Month A daily log of the CVS history files for all Codendi projects. The script /usr/share/codendi/src/utils/cvs1/cvs_history_parse.pl runs daily and is responsible for extracting the CVS log information of that day for all /cvsroot/ProjectName/CVSROOT/history file and assemble them all in a single log file. This global log file is later processed by the script /usr/share/codendi/src/utils/underworldroot/db_stats_cvs_history.pl which feeds the Codendi database with statistical figures about the CVS activity of each project (like the number of CVS checkout/add/commit counters appearing on the Welcome Web page of the Codendi CVS service ) . /hostname/ File name pattern: vhosts-access_YYYYMMDD.log Year/Month This file contains the combined Apache access logs for all the Codendi projects Virtual Web servers (also know as the project web sites). This is a simple daily copy of the Apache log file /var/log/httpd/vhosts-access_log. that is later analyzed to feed the Codendi database with statistical data about the project Web site activity. These files are generated daily by the logrotate script of Apache (see the Log Files section) . /Year/Mont File name pattern: http_combined_YYYYMMDD.log h/ This file contains the combined access logs to the main Codendi Web site. They are generated daily by the logrotate script of Apache (see the Log Files section). It is basically a copy of the /var/log/httpd/access_log file
Slide 12: File name pattern: ftp_xferlog_YYYYMMDD.log This file contain the ftp access logs (upload and download) to the Codendi site. It is generated daily by the logrotate script of ftp (see the Log Files section). It is basically a copy of the FTP log file /var/log/xferlog. Dump files Location Description /var/lib/co As we will see later in this document there is a bunch of background script running on Codendi dendi/dum on a regular basis. The default frequency if once every other hours. The first thing these scripts ps do is to extract a number of information from the Codendi database and dump in a various text files to exploit them in the next stages. Files found in this directory are • aliases: all Codendi users have an email alias in the form of login login_name@users.your.codendi.domain. By creating such aliases one can send an email to a Codendi user simply by using it's login name. This is very convenient if the project admin wants to send e-mail to certain developers of the team through CVS notification for instance. By using the login name Codendi users also make sure that if the email address of a user changes it will be automatically updated. All user aliases and mailing list aliases are actually appended to the master file /usr/share/codendi/src/utils/underworld-dummy/aliases.zone. This file contains codendi wide aliases like codendi-admin or codendi-contact. If you want to add more aliases modify this file NOT the /etc/aliases... files subversion_dir_dump ready to use Apache config files with a list of all the per project subversion repositories. This will be copied as is in the /etc/httpd/conf.d/codendi_svnroot.conf file. db_top_groups_calc_output: contains all sorts of statistics about the Codendi hosted projects (project name, old project rank in terms of total number of downloads, old project rank in terms of number of downloads this past week, user rank - unused-, rank in terms of total number of forum posting this past week, total number of download, new rank of the projects in terms of number of downloads,... group_dump: group (project) name dump with project ID, users belonging to the project and the project status (Active, Suspended, Deleted) list_dump: dump of all the mailing lists created by the Codendi projects along with the email of the administrator and the default admin password for the list. ssh_dump: dump of the SSH shared keys entered by the users. Shared keys will be automatically installed in the user home directory to let her login through ssh without having to type her password. user_dump: a user info dump with user id, status (Active, Suspended,...), default shell, MD5 encrypted password, and the real name. • • • • • •
Slide 13: User directories Location /home/users /... ...UserNam e Description Each registered user has her own Unix account created and the home directory that goes with it. This user home directory is owned by UserName.UserName and has access permission rwxrxr-x. This can be changed by the user if she wants to set access permission differently. There is currently no space quota on a user home directories. Project Directories Location Description /home/group/. Each Codendi hosted project has its own project directory. When this directory is initialized a .. minimal directory structure is created to host the Project Web Site. Two sub-directories are ...ProjectNa created: me • htdocs : this is where all html, shmtl, php,.... file should go (Apache Virtual Server Document Root) • private : this is a private directoy that is only accessible to project members These directories all have the group setuid bit set. On Linux this ensures that files created under this directory by a group member are automatically created with the right group ownership. The reason for that is because a user can belong to several Codendi projects and therefore he can be a member of several Unix groups. If the setuid bit wasn't used it would be the responsibility of the user to change its working group with the newgrp command when going from one project directory to another. This would be of course error prone. File Release Space
Slide 14: Location Description /var/lib/codendi/ft This is where all the uploaded files associated with a given release of project p/codendi/Project ProjectName (see the Codendi File Release Service) are actually stored. Normally neither Name the Codendi Administration Team nor the Project Admin Team should mess up with this area by hand using their shell account. Files are deposited here by the fileforge utility. This directory belongs to the project group ID of course and is group writable. /var/lib/codendi/ft The FTP incoming directory is used by the project members to upload the software p/incoming packages to be associated with new software releases in the Codendi File Release Service. This is a temporary storage area as the uploaded files move to the final release space /var/lib/codendi/ftp/codendi/ProjectName (see above) when a new software release is created. Remark: There is a script in the root crontab that removes all the files older than 2 weeks from this directory. The reason we do that is because this zone is shared by all the projects and we don't want that ghost files stay here for too long. FTP Space Location Description /var/lib/codendi/ftp/ This is the anonymous FTP space assigned to project ProjectName at creation time. It pub/ProjectName can be used by the project team for any purpose (well... of course it must have something to do with the project). Ownership is dummy.ProjectName and permission drwxrwxr-x) Codendi Databases The Codendi Master Database Codendi uses the MySQL database engine to store all of the project data. MySQL is fast, lightweight and very robust.
Slide 15: Database Creation The Codendi database is created once when the Codendi software is first installed. The scripts used to create and populate the database are located in src/db/mysql. Remark: NEVER EVER run these SQL scripts once the DB has been created or it will destroy all of your existing data !! Database Update As you operate your own Codendi site, the Codendi team at Xerox will provide you with a number of feature enhancements and defect corrections that you may wan to install on your site (see the Codendi Code Update section below). Some of the changes may impact the database structure in which case a database patch named dbxxxx.sql or dbxxxx.pl will appear under src/db/upgrades. Database Access Permissions MySQL has its own access permission management system that is completely independent from the access permission management system of the underlying operating system . Although we use the same user names for the MySQL and Linux administration accounts they are actually managed separately. Although this is not mandatory, we suggest that you use the same password on Linux and MySQL for a given account. The Codendi administration accounts with access to the Codendi database are as follows: • • • • root: This user has full administration right on all the MySQL databases and users. You must use this user whenever you want to grant/revoke new permission to a given user or network domain, etc... codendiadm: the user that was specifically created to have read/write access to the codendi database. It can only interact with the database from the local host and not from a remote machine. All the Codendi PHP scripts run under this user name to connect to the database (see the file /etc/codendi/conf/local.inc for the configuration of the database user and password). dummy: this user was created with read only access on the Codendi database (SELECT granted). It has no password and can only be used for a connection from the local host. This user is the one used by all Project Virtual Web site and if they include PHP scripts they can connect to the DB and extract information about their project as well as user/login/session information. cxuser: this user has read-only access to the project databases that are created when a project administrator create a project database export (See the Project Data Export section in the Codendi User Guide). No password is needed for this user. Interacting with Databases Whatever way you decide to use to access the Codendi databases (command line or phpMyAdmin), reading (part of) the MySQL documentation is a pre-requisite. Don't forget that you are now entering the realm of the core project data and making any mistake here is NOT allowed or you'll probably cause SEVERE damages. Command Line From any Unix account, type:
Slide 16: $ mysql -u user_name database_name -p where database_name is the name of the database you want to connect to (e.g. "codendi" for the Codendi master database) and the user_name is the MySQL user name you pretend to be (type -p if there is a password associated with this user but don't type the password in clear here to avoid having the password appearing in the shell command history file) Then the database is yours and you can interact with it through using SQL statements. Here are two sample SQL statements mysql> SELECT group_name FROM groups WHERE is_public = 0; # show a list a private project name mysql> UPDATE user SET realname='Laurent R. Julliard' WHERE user_id = 103; # update the full name of the user_id 103 ** IMPORTANT REMARK ** It is highly recommended that you practice a bit on a MySQL test database before playing with the real Codendi database. Ideally you should always test a SQL statement on the Codendi test server before running it on the production server. Except for SELECT statements which are harmless anyway. phpMyAdmin phpMyAdmin is a PHP based graphical User Interface to the MySQL database. It is currently configured on the Codendi production server to give access to the Codendi database only. You can access phpMyAdmin at http://your.codendi.domain/phpMyAdmin This is password protected at the HTTP level. Authenticate with the codendiadm user name and password. Apache The Apache master configuration file is in /etc/httpd/conf/httpd.conf. The master file also includes specific configuration files located at /etc/httpd/conf.d. Master server The master Codendi Web server can be accessed through one the following URLs: xww.your.codendi.domain, www.your.codendi.domain or simply your.codendi.domain. and any aliases that you may have defined yourself in the file /var/named/codendi.zone. The Apache Document Root is at /usr/share/codendi/src/www where all the PHP scripts are located.
Slide 17: Virtual Servers Each project has its own project Web server. These servers are configured with a virtual host alias in /etc/httpd/conf/httpd.conf. CGI scripts are disabled, but PHP is available, with a few restrictions (open_basedir, and safe_mode). FTP The FTP server configuration file is in /etc/vsftpd/vsftpd.conf and it has been configured to work with Codendi. The default configuration allows access to both anonymous users (using the "ftp" account) and to all registered users through their normal Codendi login/password. This can be used to edit the HTML pages of the Project Web server pages remotely for instance with Netscape Composer, DreamWeaver, FrontPage and many other Web editing environment CVS Using CVS on Codendi This document is not a tutorial on CVS. For an introduction to CVS refer to the CVS section of the Codendi User Guide. Codendi specific CVS The CVS RPM packages installed on Codendi contains a modified version of cvs. The reason the CVS source code has been modified is as follows: When cvs runs in client server mode (as it does on Codendi) the server must be told which directories are allowed as CVS root directories. These are the various places where the CVS repositories are located. Allowed CVS repositories are given to the CVS server through the --allow-root=pathname command line options where pathname is the allowed path. This option is repeated as many times as the number of CVS repositories. While this way of doing is OK for a small to mid-size CVS server it doesn't scale to hundreds or thousands of CVS repositories because repeating --allow-root options hundreds of times would exceed the maximum allowed size of a Linux command line. Therefore, a new option has been implemented in the cvs server called --allow-root-file=filename where filename is the name of a file containing the list of all allowed CVS repositories on Codendi. On Codendi this file is created/updated by the daemon script and resides in /etc/cvs_root_allow. Remark: as a Codendi customer you will automatically receive any new version of the Codendi specific CVS RPMs and you don't have to apply any of the patch by hand to the Codendi sources.
Slide 18: Running the CVS server The CVS server does not run permanently in the background. It is launched only when necessary through the inetd daemon that is configured to listen on the port of the pserver protocol (tcp/2401). Look for /etc/inetd.conf/cvs for the command line used to run the cvs server. DNS Codendi has its own internal domain : codendi.example.com. And the codendi machine is the name server for this domain. The DNS configuration files are located in: • • /etc/named.conf: top configuration file /var/named/chroot/var/named/codendi.zone: this is where the codendi.example.com domain is defined. Static configuration file The top configuration file is in /etc/named.conf. This file is static and must be edited by hand. Before the Codendi server is installed you must ask your network administrator to create the Codendi domain your.codendi.domain and delegate the administration of this domain to the Codendi server. The Codendi zone is defined in /var/named/codendi.zone . It contains the IP address for the master Codendi server as well as a list of aliases defined like www, cvs, download, shell, users, lists, etc.... The reason why all these aliases have been defined is because as the load on the main Codendi server increases it may be desirable to host some Codendi functions on a separate server (e.g the mailing list manager or the CVS server). Doing so will just be a matter of changing the IP address associated with the lists.your.codendi.domain or cvs.your.codendi.domain and all the rest will be transparent for the end users. Similarly mail exchange records (MX record) have been defined for lists.your.codendi.domain. Strictly speaking this is not necessary today because the IP address of the machine receiving the e-mail is the same as the one lists.your.codendi.domain is pointing to. But in the future we may want to have a dedicated server to handle email for the codendi domain. Hence the MX record. Wildcard DNS record As explained above each project on Codendi has its own Web server that can be accessed at the URL http://projectname.your.codendi.domain, as well as dedicated cvs.projectname and svn.projectname domains. For this to work, Codendi now provides a wildcard DNS record that accepts all subdomains. It it defined in codendi.zone with these lines: (Xerox server) ; ; Wildcard DNS entry, to match all possible hosts: projnamme.*, cvs.projname.*, svn.projname.*, etc. ; * IN CNAME atlas
Slide 19: SSH The Secure Shell service is available on Codendi. All registered user with an active account can use it to login into Codendi in a secure way. To make Windows users life easier you may also activate the telnet service as telnet comes standard with the Windows operating system. However we highly recommend not to enable telnet for security reasons and instruct your Windows users to install an SSH client instead. As a Codendi site administrator you may also decide to disable shell accounts for all users. If this is so you must change the default shell assigned to new Codendi users when they register. See the SSH section in the Codendi installation Guide for detailed instructions. ** IMPORTANT REMARK ** For security and traceability reasons, it is recommended that Codendi administrators never use telnet to log to the Codendi server. It is not secure and it makes critical password travel on the Xerox network in clear text!! The Codendi team recommends that Codendi administrators follows the following procedure to log into the Codendi server: 1. Connect to the Codendi server using ssh and your personal Codendi account (e.g. ssh username@codendi.example.com) 2. Then if you must act as root or codendiadm user use the su command (su - root) and type the appropriate password The reason for the 2-step process is to be able to determine from the /var/log/messages log file who used a privileged user account in case there are multiple Codendi site administrators. Sendmail By default Codendi uses sendmail as its mail transport agent of choice to handle incoming and outgoing mail messages. The critical files for the sendmail configuration are: /etc/mail/sendmai The master configuration file. It is created by the Codendi installation script. l.cf /etc/aliases This is a small aliases file where you only want to configure system wide aliases like postmaster, webmaster,etc... When you modify this file run the command "newaliases" for the changes to take effect. /etc/aliases.code This file is generated automatically by the Codendi daemon scripts and must never be edited by hand. The beginning of the file contains some predefined ndi aliases (codendi-admin, codendi-contact, noreply,...) that must not be modified.
Slide 20: Mailman The Mailman mailing lists manager requires little attention from the Codendi administrators. The configuration is done at installation time and mailing list are entirely managed by the end users through the Mailman Web interface. Mailman has a super user password allowing Codendi site administrators to access the administration interface of any mailing lists created by Codendi projects. OpenFire The OpenFire XMPP/Jabber server requires little attention from the Codendi administrators. You may need to synchronize the OpenFire server with the Codendi DB in some cases (e.g. the OpenFire server was down for some time). Symply go to the Codendi Administration page and select the "Instant Messaging" link. The OpenFire administrative interface is available from the main Codendi administration page. By default, only the 'admin' user (same password as for Codendi) has access to the OpenFire administration interface, but other users can be added easily. Please read the JabbeX Administration Guide for more details. Background daemons The root crontab The root user on Codendi has a number of script defined in the crontab. As the name says it this the master script that compute all the /usr/share/codendi/src/utils/ daily statistics about the Codendi site globally and also for compute_all_daily_stats.sh individual projects /usr/share/codendi/src/utils/ underworld-root/... Same as above but this time it is for weekly statistics db_project_weekly_metric.pl
Slide 21: /usr/lib/codendi/bin/backup_j Backup of all Codendi file system. ob /usr/lib/codendi/bin/backup_s Backup of all Codendi subversion repositories. ubversion.sh Log Files Log Files For a description of the Log files used by the Codendi server see the Log Files section above. In addition to these files there are a number of other log files that are worth knowing: /var/log/mess This is the general purpose log files used by the Linux operating system. What goes in there is defined in /etc/syslog.conf ages /var/log/mail This is where all the log messages related to mail activities go. All sendmail messages log for message sent and received are stored in this file. Log file rotation Linux has the ability to automatically rotate the log files to prevent them from growing indefinitely. The logrotate utility is ran everyday at 23:58pm (EST time) as defined in /etc/crontab. Note that this is different from the crontab file of the root user that is used to run the Codendi daemon scripts. The logrotate configuration files located in /etc/logrotate.d/apache and /etc/logrotate.d/vsftpd.log have been customized for Codendi. The customization consists in the execution of a post rotation script that makes a daily copy of the log files in the Codendi log files area as described in the Log Files section. Codendi Code Update Codendi offers a continuous update process to Codendi site administrators to quickly and easily put in place new bug fixes, new features while minimizing the server downtime. The Codendi software is made of PHP and Perl code that is delivered in source form to our customers. The Codendi software resides in the /usr/share/codendi directory and it is formatted in such a way that it behaves like a Subversion
Slide 22: working copy that can be remotely updated when ever the system administrator decide to do so. As a Codendi customer you will be entitled to receive the Codendi Subversion events by email. They are reports about the defect that have been solved and the new feature that have been implemented in Codendi. Benefiting from these new features is just a matter of update your Codendi working copy. Codendi offers two ways for updating your server : • • Classic update in console environment Web interface update mechanism Classic update in console environment Here is how you should proceed: • First you should make sure that the Codendi source code repository that you must use for your updates is accessible. Codendi Site Location Codendi repository pointer Customer Extranet https://partners.xrce.xerox.com:/svnroot/codendi • • • Log to the 'codendiadm' Codendi shell account on the Codendi server Go to the Codendi top directory: cd /usr/share/codendi Launch an SVN status command to get a list of all the files that needs to be updated on your local Codendi directory svn status -u --username username (where username is your login on the Codendi support site given to you when your support contract was activated) • At that stage you must carefully review the file list and look if the files src/etc/local.inc.dist, src/etc/httpd.conf.dist or src/db/upgrades/dbxxxx.sql are mentioned in the list. If this is so go to the next step. If not then it 's a normal update operation that you can do with a simple: svn update --username username ... and that's all you have to do!! • If src/etc/local.inc.dist is in the list of files to be updated then you must look at the differences with your own local.inc.dist and make the changes in your /etc/codendi/conf/local.inc file accordingly. To see what has changed in the local.inc.dist file type: svn diff --username username src/etc/local.inc.dist # look at the changes and keep them in a safe place svn update --username username # do the real update vi /etc/codendi/conf/local.inc # edit your Codendi
Slide 23: configuration file and merge the changes then do the real update as above. • • Similarly, if src/etc/httpd.conf.dist is in the list of files to be updated, you should look at the differences with your own httpd.conf.dist, then merge them in your /etc/httpd/conf/httpd.conf. If one or several src/updates/scripts/XXX_<script_description>.class.php (where XXX is a three digit number) are listed then it means you have to perform some script update (on the Codendi database structure for instance). You must first do the real update and then apply each update script in order starting with the smallest XXX number first and growing svn update --username username # do the real update cd src/updates; sh upgrade.sh XXX_<script_description> apply the script XXX_<patch_description>.class.php # Web interface update mechanism Another way to update your working copy is to use the plugin ServerUpdate. This plugin provides a secure and easy-to-use process to update your server, with web interface overview and lots of feedback about what happens. This section will focus on the automatic update process. For a user guide, please read the on-line plugin documentation. Process one or more updates The plugin allows you to update automatically the Codendi server through the Codendi web interface. The update is always processed from the current revision of the working copy to the chosen revision (greater than the current revision), by updating all the revisions between. To update the server, click the icon on the right of the wished revision. The update will be processed in two steps: 1. Update simulation • The system simulate the update, checking that: • there is no manual intervention required, • there is no subversion conflict, • there is no scripts conflict. 2. Real update • If no conflict was detected during the update simulation, • After the administrator confirmation, • Incremental update (one revision by one) • Success test for each revision, • Action feedback given to the user, for every action done for each revision, • Details about upgrade scripts executions if so.
Slide 24: Update Strategy Simulation strategy: The system first checks if there is no manual intervention required between the current revision and the requested update. A manual intervention prohibits the update initially. It is indeed possible to enforce the update, but we would draw your attention in the danger of such an operation (see section Force mode for details). Then, the system checks there is no subversion conflict. For doing this, the system simulate a global update from the current revision to the requested revision. (by using the subversion command svn merge -dry-run -r <working copy revision>:<requested revision> <subversion repository> <working copy directory>). The system then checks there is no upgrade script conflict. An upgrade script will be executed if and only if: • • • The script is added in the requested update and It is not deleted in a later revision and It is not modified after the last revision of the requested update. In the case where the script would be modified, the only execution of the script will be processed during the last modification. An upgrade script will not be executed if: • • • This script is deleted in a later revision. In this case, there is two options: • the requested update goes up to the script deletion: in this case, the script is just ignored (not executed), and the update is possible. • the requested update doesn't go up to the the script deletion: in this case, the update is not allowed. Because if a script is deleted, it is possible that the revision between keep the server in an unstable state. There is so no other way but to update up to the script deletion. This script have been added before the current revision (the first appearance of this script in the requested update is a modification or a deletion). This extremely rare situation could happened if a script is wrong and if you updated your server before the fix. In this case, the update is not allowed. We would manage the situation with individually. This script is modified in a later revision not included in the requested update. In this case, it is obligatory to process an update up to the last script modification. Update strategy: After a simulation and after the update confirmation, the system will almost use the same strategy, except for the subversion update. Indeed, the update is done incrementally, analysing each time the subversion return for detecting the potential errors. The subversion command used is the following one: svn update -r <requested revision> Manual intervention and Force mode: Some updates may need special interventions, that would be better manually done. It would be operations to do in addition of the simple working copy update, such as a file system modification, needing root
Slide 25: permissions for example, or also an intervention that would need a temporary server stop). In this case, the revision is labelled as needing a manual intervention and as we saw it explaining simulation strategy, such a situation is stopping regarding the automatic update. This to catch your attention about the special type of the update and to make you read the commit message that contains information about the the update. However, it is possible to enforce the update, but we wish to catch your attention about the danger of such an operation. If you don't respect the indication given in the commit message of such an update, you could find your server in an unstable state. So, only use the "force" mode only if you are sure of what you do. Backup/Restore Backup There are three different types of backups that run on Codendi. First, MySQL data is backed up to disk using the linux "dump" command. Second, the Subversion repositories are backed up with the backup_subversion.sh script. And finally, all disk partitions are backed up to tape using Legato software. 1. MySQL data - A cron job is scheduled to run every Sunday morning at 00:45. MySQL data is backed up to disk. The sequence is as follows: /etc/rc.d/init.d/mysql stop • move old codendi-bin.index, codendi-bin.001, sqldump.<last week's date/time> to /var/lib/codendi/backup/mysql/old directory • dump 0f /var/lib/codendi/backup/mysql/sqldump.<date>.<time> /var/lib/mysql • /etc/rc.d/init.d/mysql start - reads /etc/my.cnf - new codendi-bin.index created - new codendi-bin.001 created The whole MySQL data backup process takes less than 15 seconds • 2. Subversion repositories: a specific subversion backup script (/usr/share/codendi/utils/subversion_backup.sh) is run in the root crontab. The script offers two backup mechanisms: full backup, and incremental backup (with '-i' flag). Currently, full backups are run weekly, while incremental ones are run daily. 3. Disk Partitions - Using Legato software, a full backup of all disks (except /tmp) is done beginning at approximately 01:00 every Sunday morning. It is run after the MySQL data is backed up to disk so that the MySQL dump file goes to tape. All remaining days of the week (Monday - Saturday) run incremental backups. Database Restore 1. 2 files are needed to rebuild. Find the most recent "dump" and "bin" files:
Slide 26: cd ls /var/lib/codendi/backup/mysql -lt Example: sqldump.080502.004501 (created from cron "dump" every Sunday; mysql is shut down prior to "dump") service mysqld stop cd /var/lib mv mysql mysql.bck 2. Then restore the mysql databases from sqldump.080502.004501. tar -vxf codendi/backup/mysql/sqldump.080502.004501.tar 3. Then, change the ownership and the security context of the directory: chown -R mysql:mysql /var/lib/mysql chcon -R system_u:object_r:mysqld_db_t /var/lib/mysql 4. Finish the mysql rebuild by reloading data from codendi-bin.000001, which contains changes since sqldump.080502.004501 was created. Boot system in single user mode: We don't want anyone trying to update the mysql database before everything has been restored. Before reloading date, you have to update /etc/my.cnf : Replace : [client] default-character-set=utf8 by : [client] loose-default-character-set=utf8 And then : service mysqld start cd /var/lib/ /usr/bin/mysqlbinlog mysql.bck/codendi-bin.000001 password: (enter root password) | mysql --user=root -p No information will display until it's finished. Rebuild is complete. Backup/Restore files There are a number of files used for backup and restore of Codendi. Location Description /var/lib/codendi/back The three most current files found in this directory are: up/mysql
Slide 27: • • • sqldump.<date>.<time>: dump of /var/lib/mysql codendi-bin.index: Text file indicating file name to use when restoring mysql data codendi-bin.001: Binary log file used for viewing or piping into a mysql command if rebuilding database Older files are stored in /var/lib/codendi/backup/mysql/old /etc The file found in this directory is: • my.cnf: This file is read when mysql starts. MySQL variables are set here. /usr/lib/codendi/bin The files found in this directory are: • • backup_job: The script that runs from cron at 00:45 every Sunday. backup_subversion.sh: The script that performs incremental and full backups of the subversion repositories. /etc/rc.d/init.d Codendi disks are backed up using Legato software. There are 2 nsrexecd daemons running at all times. The script found in this directory is: • networker: the script that runs at boot time Codendi Testing Environment The Codendi source code has been modified so that multiple instances of the Codendi software ca run on the same physical machine. The goal here is to allow each developer in the Codendi Team to have his/her own source tree available for modification and testing and avoid any interference with other developers. The setting of a developer's testing environment is fully document in the Codendi Configuration Management Plan. Codendi HowTo
Slide 28: Importing an existing CVS repository in Codendi How to properly install a project team's existing CVS repository into the Codendi project specific repository Preamble The main thing to understand about Codendi is *each* project has its own CVS repository. This root dir is created and initialized by Codendi whenever a new project is registered. Its name is "/cvsroot/projectname" I insist on this point because most of the time in many organizations there is one single CVS repository where all projects are stored. The problem with this typical organization is that the directory named "CVSROOT" which contains CVS config and admin files is unique and global to all projects managed by CVS. The directory named "CVSROOT" contains files such as "modules" to declare module name aliases, or commitinfo,rcsinfo and loginfo to trigger specific actions when commit happens (such as e-mail notification to certain people,etc.), or taginfo to put constraints on the TAG formats, or readers and writers which are 2 files to allow/deny access to the CVS repo. If we had one single CVS repository on Codendi then the unique and global "CVSROOT" directory would rapidly become a big mess and it would be extremely difficult to maintain all the project specific configurations in this single "CVSROOT" directory. Hence the decision to have one CVS repository per project on Codendi. The real work 1. Register the project. Codendi Admin must approve it and wait for the 2 hours cron update to create a fresh /cvsroot/projectname CVS repository with a fresh CVSROOT directory in it. If you are in a hurry you can force the cron update by hand. 2. Ideally the tar file provided by the team should provide both the CVS top directories *PLUS* the "CVSROOT" directory from the original CVS repository. 3. Having the CVSROOT in the tarball is not mandatory but if the project had any specific settings in any of the CVS administration file (see above) then there is no chance that the Codendi admin can rebuilt the same environment on Codendi. In practice very few projects actually use these admin files but you can't never know hence the request for the CVSROOT directory to be included in the tarball. One file that is always used though is the CVSROOT/history file where all the actions like commit, add, checkout, etc. are automatically logged by CVS. If the team wants this file to be preserved then again the CVSROOT has to be provided 4. Create a temporary directory: cd /tmp mkdir projectname cd projectname 5. Unpack the cvs tarball provided by the project team in this temporary directory: tar xvfz projectname_cvstarball.tgz 6. Assuming that the structure is now as follows /tmp/projectname - CVSROOT - history (as well as ,v files) - modules - commitinfo - ..... - top_level_dir1
Slide 29: - top_level_dir2 - .... - top_level_dirN 7. Move all top project dir to the project specific repo: mv top_level_dir* /cvsroot/projectname/ Caution!! Leave the CVSROOT directory in place for the moment 8. Go to the project CVS repo: cd /cvsroot/projectname 9. Change the ownership of the entire CVS repo: chown -R projectadminname.projectname top_level_dir* where projectadminname is the Codendi login of the project administrator 10.The CVS admin files in the CVSROOT directory must be examined one by one and the project specific instructions must be merged with the files by the same name in the CVSROOT directory created by Codendi. • **Never touch** the CVSROOT/readers, writers, passwd and config files created by Codendi • Be careful not to overwrite what is already in the CVSROOT/loginfo and val-tags file. Append the content of these given files at the end of the corresponding ones generated by Codendi. • For the history file : extract the lines belonging to the hosted project from the history file provided by the project team (remember it is very likely that this file is global to many projects) and update the content (especially the name of the directories and path listed in the history file if they are not the same on Codendi) • For all the other files, look at them one by one and see if there are project specific settings in it. If so append the settings at the end of the corresponding Codendi files. 11.Delete the temporary directory: rm -rf /tmp/projectname 12.Ask the project team to perform a checkout as explained on their CVS project page to check that everything is working fine. That's all folks! Importing an existing Subversion repository in Codendi How to properly install a project team's existing SVN repository into the Codendi project specific repository Preamble As for CVS, it is important to understand that *each* project has its own SVN repository. This root dir is created and initialized by Codendi whenever a new project is registered. Its name is "/svnroot/projectname" The real work 1. Register the project. Codendi Admin must approve it and wait for the 2 hours cron update to create a fresh /svnroot/projectname SVN repository. If you are in a hurry you can force the cron update by hand. 2. The project team should provide a dump of the existing Subversion repository. To create an SVN dump please use the following command (Unix/Linux): svnadmin dump /path_to_svn_root > svn_dumpfile
Slide 30: 3. Ask the team to copy the dump file on the codendi server, e.g.: scp svn_dumpfile username@codendi.example.com:/home/groups/projectname (it is also possible to use FTP instead). 4. then a Codendi site administrator needs to load the repository content into the existing repository on Codendi: svnadmin load /svnroot/projectname < /home/groups/projectname/svn_dumpfile 5. Then, don't forget to set proper Unix ownership on the repository: chown -R codendiadm.projectname /svnroot/projectname 6. if the existing repository had specific permissions or hooks, it is now time to copy the corresponding files on the new repository. This can be done by the project team. 7. Once the team has tested the new repository, you can remove the dump file from /home/groups/projectname/svn_dumpfile. You're done! Regenerating a fresh CVS or SVN repository for a Codendi project When the first import goes wrong or the project team screwed up everything and want to start again from scratch The fastest and safest way (where xxx is the project name): • • • • • • Log as root cd /cvsroot for a CVS repository, or cd /svnroot for a SVN repository Make a backup copy of the current tree in /tmp just in case...: tar cvfz /tmp/xxx_repository.tgz ./xxx Delete the CVS or SVN tree: rm -rf xxx Force the cron update process to run now. Delete corresponding entries from cvs_checkins, cvs_commits and cvs_descs tables to prevent showing them while browsing commits with viewVC. Now you have a fresh CVS or SVN repository and the team can start the import again. Changing a User Name on Codendi How a Codendi administrator can change a user's login name • • • • Log as root on the codendi server First of all, make sure that the new login name is not already used! Second of all, make sure that the Xerox cron job won't run while you do the update (you may edit the root crontab if necessary)! Rename the user home directory:
Slide 31: • • • • • • • • • mv /home/users/old_name /home/users/new_name Edit each of the CVS writers file for the projects the user belong to and modify the name. You can find all the relevant writers file by typing the following Unix command: grep -rsl old_name /cvsroot/*/CVSROOT/writers Edit Subversion permission files for the projects the user belong to and modify the name. You can find all the relevant files by typing the following Unix command: grep -rsl old_name /svnroot/*/.SVNAccessFile Use phpMyAdmin, browse the "user" table, look for the row corresponding to the user, edit the row and modify the name. Alternatively you can start the MySQL shell and type the following SQL statement: update user set user_name='new_name' where user_name='old_name'; Do the same for the email column in the artifact_cc table. Or use the following SQL statement: update artifact_cc set email='new_name' where email='old_name'; Do the same for the addresses column in the artifact_global_notification table. Or use the following SQL statement: update artifact_global_notification set addresses='new_name' where addresses='old_name'; Please note that it will only update the address if it is not part of a list of addresses. To check if it is part of a list of addresses, use: SELECT * FROM artifact_global_notification WHERE addresses LIKE '%old_name%' and update entries if any. To keep wiki preferences and notifications, rename the user wiki page (make sure the new page does not exist yet): update wiki_page set pagename = 'new_name' where pagename = 'old_name'; Legacy tracker only: Do the same for the from_email column in the support messages table. Or use the following SQL statement: update support_messages set from_email='new_name' where from_email='old_name'; Legacy tracker only: Do the same for the email column in the bug_cc table. Or use the following SQL statement: update bug_cc set email='new_name' where email='old_name'; Legacy tracker only: Do the same for the email column in the project_cc table. Or use the following SQL statement: update project_cc set email='new_name' where email='old_name'; Remark: if the user has already committed changes in the cvs repository, then the old name will still appear in the CVSROOT/history file and the various *,v files (CVS version files). Changing the various "history" files is possible but changing the *,v files is perilous and frankly it is is not necessary. Changing a Project Short Name on Codendi How a Codendi administrator can change a project short name. The project short name is defined a registration time. It is also known as group name because it each project has a Unix group of the same name on the Codendi server. Here is the procedure to follow to change a group name (assuming the old project name is 'xxx'):
Slide 32: **IMPORTANT NOTE** Before you start the procedure below make sure you have enough time to complete the task before the next cron job runs. • • • • • First log as root Change the old project name 'xxx' in the following directories: • /var/lib/codendi/ftp/pub • /var/lib/codendi/ftp/codendi • /home/groups • /cvsroot • /svnroot Edit /etc/cvs_root_allow and change project name Comment: the last one would be done automatically at the next cron update but doing it by hand keep everything consistent with the next step Checkout the CVSROOT directory of the project: cvs -d:pserver:username@codendi.example.com:/cvsroot/projectname co CVSROOT. Then edit the 'config' and 'commitinfo' files and change the project name wherever it appears. Last, commit the changes. (you may also directly edit those files in the /cvsroot/projectname/CVSROOT directory, but don't forget to also edit the ',v' files then). Edit the 'groups' table of the Codendi DB with phpMyAdmin and change the name (2 occurrences in 2 columns: unix_group_name, http_domain). Then edit the 'service' table, and change the link to the 'Project Summary' and potentially to the 'Project Home Page'. All other fields can be changed by the project admin via the Web interface. Remark: If there are mailing lists with existing content in the archive it is not clear for the moment how to rename the mailing list. So in this case it probably means creating new (empty) mailing lists for the new project. Of course one can still keep the ones with old project names in it. This is no problem. See additional hints there:https://www-s.acm.uiuc.edu/wiki/space/Rename+Mailman+lists. And voilà! New user registration problems - email address errors What to do when you get a message from postmaster with an attached new user registration confirmation message It is fairly common for a new Codendi user to enter the wrong email address when registering. When Codendi sends the registration confirmation message to the new user, the message bounces at whatever mail server owns the non-existent address, and a message is sent back to the originator (aliased to the Codendi administrators) saying that the message could not be delivered, with the registration confirmation message attached. What we then need to do is forward this to the user who is trying to register on Codendi, with an explanation of what happened and what the user needs to do next (complete the registration by following the link in the attachment). We may also decide to fix the email address for the user, if we know what it should be. But first you have to track down the user. (You can't just send mail to the address in the message. I mean, it didn't work :-).) Your main sources of information are the Xerox Outlook/Exchange Address Book and Codendi itself. • The mistake most users make is on the part of the mail address after the '@'. Since our Outlook mail addresses are always of the form firstname.lastname@something.xerox.com, it is usually easy to look up 'lastname, firstname' in the Outlook Address Book. Opening the entry brings up a dialog box with
Slide 33: • • • • an E-mail Addresses tab. Selecting this tab will show all of the smtp addresses for the user. In order to fix the address on Codendi, you must find the new user. As an administrator, the way to do this is to follow the User Admin link from the Site Administration list in the column on the left. (You can't see this list unless you are a site administrator.) The User Admin page, http://codendi.example.com/admin/userlist.php, displays an alphabetical list of Codendi/Unix user name links. Each user name begins a row of links, including DevProfile, Activate, Delete, and Suspend. We are interested here in the user name link itself and the DevProfile link. You have to guess what the user's Codendi user name is, based on the user's real name, which is hopefully in their mail address. Most people are accustomed to using their last names, the first letter of their first name followed by their last name, their first name followed by the initial of their last name, their first name only, etc. John Smith is probably smith, jsmith, johns, or john. User names for which registration is incomplete are shown in non-boldface with a preceding asterisk. If you see a "*jsmith" entry in the user list, you have probably found the user, in our example. If John Smith has decided to make his user name 'gandalf' (because there is already a jsmith), then you have a problem. Worst case is that you need to process the user list, looking at all of the names preceded by an asterisk, as described below. In order to look at the attributes of a user name, just follow the link. You'll see a page with the user's shell (/bin/bash by default), Unix account status, and email address. If you have found the right user, the email address is the one that failed, same as in the postmaster message sent to you. You could fix it now, if you know the right address, but note that many people in Xerox, particularly in XR&T, have multiple SMTP addresses, and preferences for one over another. If you have any doubt about what to do, turn the problem over to the user, by explaining in your note to the user how to change his/her address. (Follow the Account Maintenance link in the left column and then follow Change Email Addr.) Sometimes you want to follow the DevProfile link from the user list, in order to verify the user's real name. You could also just Activate the user now, instead of requiring them to follow the link in their account registration confirmation message, but it seems better to engage them. The mail address has to be fixed, and they should be aware that it was wrong and either correct it or consciously agree to the fixed version. I once 'helped' someone by fixing the address, and they changed it back to its original form, and complained that the Codendi mail service should have been able to find it. (True. There was a problem with Codendi mail at that time.) John Stidd Blocking a Client from Accessing Codendi How to block a client from accessing http or other services on Codendi It is possible that a client could continuously try accessing Codendi, causing unwanted activity. This could cause the /var/log/httpd/error_log or access_log to continually grow with errors, which in turn can fill up the root partition. There is a way to block the client from accessing Codendi until the owner can be contacted or the cause of the continual access can be cleared up. To deny http access to a client: 1) Modify /etc/httpd/conf/httpd.conf:
Slide 34: Example of how to deny http access to 13.0.33.210: Look for the section of httpd.conf that contains the first 4 lines shown below and add line 5: # Controls who can get stuff from this server. # Order allow,deny Allow from all Deny from 13.0.33.210 (line to be added) 2) /etc/rc.d/init.d/httpd restart To deny access to everything in inetd.conf (like ftp): 1) Modify /etc/hosts.deny: Example of how to deny access to 13.0.33.210: ALL: 13.0.33.210 (line to be added) Vicki Tuttle How to suspend/delete a Codendi user What to do with users who left the company or whose e-mail address is no longer valid Codendi is a system where any Xerox employee can register. When an employee leaves the company the account remains valid and the Codendi administrators discover that the user is no longer with the company when the mass mail engine is used to send an announcement to all users. The goal of this document is to document what to do with these invalid accounts. How to delete a user Project member If the user was part of a project then do the following: • • Set the Web status to 'Deleted' Set the Unix Status to 'Suspended' See below for more information on how to change these status values. To determine whether a user is a project member. Use the "Search People" box using the user e-mail or name then click on the appropriate result and see if any project is listed on the upper right corner of the screen. By doing so, the Unix account is inhibited which is what we want for security reasons but it is still here so all the files on the Codendi server marked with this user uid will still appear as being owned by this user. On top of that it is perfectly ok to suspend a Unix account as soon as we have an e-mail error for sbdy and if this person comes back to us complaining that his/her account is no longer accessible we can restore the account very quickly with no data loss at all (and ask him/her to update her e-mail address)
Slide 35: In case of doubt or if you are in a hurry and don't have time to check whether a user is a project member then always use this scenario. Actually we should probably always use this scenario to start with and then once in a year change the Unix status to 'Deleted' when we have no news from the user Before you suspend a user account have a look in the Xerox directory and see if the person has a new e-mail. If so then you might want to update the e-mail or send him/her a reminder to do it via the "Account Maintenance" page on Codendi. Non project members If a user is not a project member and we are sure s/he left the company (permanent e-mail error returned not transient) then do the following: • • Set the Web status to 'Deleted' Set the Unix Status to 'Deleted' The Unix account is deleted for real, the user home directory is deleted as well. Note: If there is only a couple of accounts to suspend/delete then going through the Web interface is the best way. If there are many users do the following: 1. Have a Codendi Web page ready and open on the Search people box. Type part of people's name in the box and determine the login name from the result list 2. Have a terminal session open on the Codendi server and connect to the database with: mysql -u codendiadm codendi -p then use the following SQL statement for Suspend/Delete scenario: update user set unix_status='S',status='D' where user_name='fmarcus'; or for the Delete/Delete scenario use: update user set unix_status='D',status='D' where user_name='fmarcus'; Recompiling a Codendi specific RPM Explain how to regenerate a Codendi specific RPM from the standard RPM 1. Log as root on a test server 2. Go to /usr/src/redhat/SPECS 3. Install the previous codendi specific RPM in source form if necessary to get the specific patches and also the specific spec file 4. Install the new version of the standard source RPM 5. Make a diff between the old codendi specific spec file and the new standard spec file to see what are
Slide 36: the changes. 6. Merge the changes you made in the old codendi specific spec file in the new spec file 7. Don't forget to append the 'codendi' suffix to the release name in the new spec file. 8. Also change the name of the new spec file to reflect the version number and the codendi prefix (e.g. cvs-1.11.1p1-8.7.spec) 9. Recompile the package (rpm -ba ... to generate both the source and the binary package 10.Install the new modified binary RPM package on the test server and make sure it works ok 11.Publish the new packages (both source and binary) in the package release space (/home/groups/codendi/htdocs/packages/RPMS_Codendi) 12.Send an email to the codendi-devel mailing list to announce the availability of the new codendi specific package and point them to the RPM release space ( http://codendi.partners.xrce.xerox.com/packages/ ...) Change privileged account passwords Steps to change privileged account passwords Some companies have specific policies regarding passwords. For instance, the Xerox Electronic Standards Security Guide (v3.0) states that all privileged account passwords must be changed at least every 30 days. See http://taweb.mc.xerox.com/Get/File-84236/ESS.doc, Section 4.3.2 Passwords, for details. (Intranet) At Xerox, passwords will be changed the first Monday of each month. 1) Change the passwords for the following accounts: root codendiadm admin mailman 2) Change the codendiadm password in /etc/codendi/conf/local.inc. (Restarting services is not necessary.) 3) Send note to codendi-admin@lists.codendi.example.com stating passwords have been changed. 4) All Codendi admins need to change the password on their privileged personal account. root account: 1) Change unix password from root account: - password (cr) [enter new password] [retype new password to confirm] 2) Change mysql password: - mysqladmin - -user=root -p password [enter new password] enter password: [enter old password] codendiadm account: 1) Change unix password from codendiadm account: - password (cr) [enter new password] [retype new password to confirm] 2) Change mysql password:
Slide 37: - mysqladmin - -user=codendiadm -p password [enter new password] enter password: [enter old password] 3) Modify /etc/codendi/conf/local.inc - change codendiadm password admin account: - The "admin" account is the first user that is created when a new Codendi site is created. - The "admin" account is used as an "administrator" for orphaned projects. - The "admin" unix account is not a privileged account. 1) Change unix and mysql password for admin from Codendi website: - Go to http://codendi.example.com - Log in as admin - Click "Account Maintenance" in left bar - Click "[Change Password]" - Follow directions for changing password mailman account: - The "mailman" account has the universal password for mailing lists. 1) su 2) su mailman 3) cd /usr/lib/mailman/bin 4) ./mmsitepass [enter new password when prompted] 5) Change unix password from root account: - password mailman [enter new password when prompted] [retype new password to confirm] - To test the mailman account: - Go to http://lists.codendi.example.com/mailman/admin - Click on any mailing list - Type new global admin password If you see the admin page of the selected mailing list, then it is OK. Delete a bug from the Codendi database Delete a bug with its given bug_id from all concerned DB tables. Some projects might ask to delete submitted bugs from Codendi because of sensitive data that the bug information contains. While the functionality "delete bug" is not integrated into Codendi you can use this little sql script to do the work by hand. SET @bug_id = your_bug_id; DELETE FROM bug WHERE bug_id=@bug_id; DELETE FROM bug_bug_dependencies WHERE bug_id=@bug_id; DELETE FROM bug_bug_dependencies WHERE is_dependent_on_bug_id=@bug_id; DELETE FROM bug_task_dependencies WHERE bug_id=@bug_id; DELETE FROM bug_cc WHERE bug_id=@bug_id;
Slide 38: DELETE FROM bug_file WHERE bug_id=@bug_id; DELETE FROM bug_history WHERE bug_id=@bug_id; Save this script for instance as del_bug.sql and launch it with mysql -u codendiadm -v -v -v -p codendi < del_bug.sql > out Enabling 'Restricted Users' on a Codendi server Enabling restricted users to benefit from code sharing without compromising security. Overview One of Codendi main benefits is to enable code sharing and re-usability in a corporation. The goal is to provide visibility of your project and code in the whole community of users. Still, in some cases, this visibility becomes an issue: • • When Codendi is used for customer interaction, e.g. to collect customer support requests and bugs. when some project members are from other companies (contractors), and should not have access to the whole site In both cases, external people (customers and contractors) need an access to the server; yet, they should not be granted access to all the other projects hosted on Codendi. A solution to this problem has already been put in place: use two servers instead of one (one for internal use, one for 'partners'). Yet, there is an issue with projects that are being developed internally and that have customer feedback. In this case, you need to duplicate the project, which is not convenient; customers cannot access commit pages referenced in commit emails (because they point to the internal server),etc. Now, there is another solution by enabling restricted users on your server: these users (typically, external users), are granted access to their project pages only, and are denied access to any other page (other projects, software map, etc.). Normal users still have access to the whole server. Configuration local.inc In order to setup restricted users, you need a coherent local.inc. Edit /etc/codendi/conf/local.inc and make sure that the following variables are properly set: $sys_user_approval = 1; $sys_is_project_public = 1; $sys_allow_anon = 0; $sys_allow_restricted_users = 1;
Slide 39: Some explanation: • • • • $sys_user_approval = 1 is needed because it is during the user approval phase that the Codendi administrator is able to set the user status to 'Restricted' (R) or 'Active' (A). If the variable is set to '0', users will be created as Active by default. $sys_is_project_public = 1 is not absolutely necessary. It just seems useless to have restricted users if all projects are private. $sys_allow_anon = 0 is needed. Otherwise, anonymous users (i.e. users who have not logged in) would have more privilege than restricted users. So a restricted user would simply need to log out to be able to browse other projects, etc. $sys_allow_restricted_users = 1 simply enables the 'Restricted' status for users. Restricted Shell By default, restricted users do not have a regular shell access: they are given a restricted shell access that only supports a few command (only 'cvs' today). The default shell is '/usr/lib/codendi/bin/cvssh-restricted'. It grants CVS access to projects the user is member of, and forbid access to all other projects repositories. If you need to completely remove shell access (and forbid CVS), you need to modify the shell manually in the administration interface for each user: set it to /sbin/nologin. Setup: in order to use CVS, the restricted user must do the following: • • Set the 'CVS_RSH' environment variable to 'ssh' use the following command line: cvs -d:ext:username@codendi.example.com:/cvsroot/projectname co module See also 'Codendi Installation Guide'. Setting restricted users privileges You may fine tune the privileges of restricted users on your system. Simply copy /usr/share/codendi/sitecontent/en_US/include/restricted_user_permissions.txt in /etc/codendi/site-content/en_US/include/ and edit it. This is a sample configuration file: // comment/uncomment forbidden URLs $forbidden_url = array( '/snippet/', // Code Snippet Library '/softwaremap/', // browsable software map '/new/', // list of the newest releases made on the Codendi site '/search/', // search for people, projects, and artifacts in trackers! '/people/', // people skills and profile '/stats/', // Codendi site statistics '/top/', // projects rankings (active, downloads, etc) '/project/register.php', // Register a new project '/export/', // Codendi XML feeds '/info.php' // PHP info ); // Use true/false for those options $allow_welcome_page=false;// Allow access to Codendi welcome page (at e.g. http://codendi.example.com/)
Slide 40: $allow_news_browsing=false; // Allow restricted users to read/comment news, including for their project $allow_user_browsing=true; // Allow restricted users to access other user's page (Developer Profile) $allow_access_to_codendi_forums=true; // Codendi help forums are accessible through the 'Discussion Forums' link $allow_access_to_codendi_trackers=false;// Codendi trackers are used for support requests on Codendi $allow_access_to_codendi_docs=false; // Codendi documents (Note that the User Guide is always accessible) $allow_access_to_codendi_mail=false; // Codendi mailing lists (Developers Channels) Other considerations • • • Restricted users must be denied access to the pserver protocol to access CVS: only the SSH method is supported (through the restricted shell). If you want to disable the pserver access, make sure you edit/etc/xinetd.d/cvs, change the 'disable' parameter to 'yes' and restart xinetd (service xinetd restart). You may also fine-tune the configuration file to allow pserver for some IP addresses and deny it for others.. Access to projects web sites by Restricted Users is not controlled. If a project web site displays sensitive data, then it should put in place access restriction mechanisms (e.g. a '.htaccess' file). Subversion: currently, if the sys_allow_restricted_user variable is set to '1', subversion repositories have their default access policy changed: by default, only project members have read access (as well as write access). If other users need to access the SVN repository, they need to be individually added to the subversion access file (through the svn admin page). Localize service names If you would like to add new system-wide or project-wide services please note the following: For those service names to be localized, we store a simple key for the service label and description in the database. These keys are then translated into the users current language by doing a look up of the key in the site_content/<language>/project/project.tab file. The keys follow a simple pattern: • • service_<service short name>_lbl_key for the service label service_<service short name>_desc_key for the service description To assure the correct localization when adding a new service please follow the instructions below: • • • Choose service_<service short name>_lbl_key as service label Choose service_<service short name>_desc_key as service description Add two entries into each site_content/<language>/project/project.tab project_admin_editservice service_<service short name>_lbl_key <your localized service label> project_admin_editservice service_<service short name>_desc_key <your localized service description>
Slide 41: Convert a CVS repository to Subversion Some projects may want to switch from CVS to Subversion. There are many good reasons for this, e.g. performance increase over CVS as well as many new features like directory and symbolic link versioning, file and directory move, truly atomic commits, etc. Unfortunately, project members cannot do the full conversion process by themselves because of permission issues. Here are the step-by-step instructions: • • • Log in as root. If not already done, install cvs2svn from http://cvs2svn.tigris.org Check that the SVN repository is empty: svnlook info /svnroot/projname • Then simply type: cvs2svn --existing-svnrepos -s /svnroot/projname --tmpdir=/tmp /cvsroot/projname • • You might need to add "--encoding=iso8859-1 --encoding=utf_8" if the conversion process stops because of character encoding issues. Note: If the conversion fails with a Berkeley DB error, this might be caused by BDB version differences between the svn client (v1.2+) and the svn repository (v1.0 or v1.1). In this case, delete the old repository, and recreate it with the backend script. Actually, you should also upgrade all existing svn repositories that use the deprecated version of BDB... This will convert the CVS repository with all the historical information (including all commits, tags and branches). To select a set of historical data, please read: http://cvs2svn.tigris.org/cvs2svn.html You should then set proper ownership on the repository: chown -R codendiadm.projname /svnroot/projname • • Activate the Subversion service in Codendi if it is disabled. You may also populate the Codendi DB with subversion revision details: you need to execute as codendiadm '/usr/lib/codendi/bin/commit-email.pl NNN' for each revision number (NNN) created. Please note however that CVS commits performed by people whose login name does not correspond to Codendi logins won't appear. Restore a Subversion repository Subversion repositories are backed up every day. So it is always possible to restore the repository as it was the day before. There are two Subversion backup directories: • • /var/lib/codendi/backup/subversion/full contains the full backups performed weekly /var/lib/codendi/backup/subversion/incr contains the incremental backups performed daily. In order to restore the most recent version of the repository, you need:
Slide 42: • • to restore the most recent full backup: simply copy /var/lib/codendi/backup/subversion/full/current/projectname-rev1 as /svnroot/projectname, and note the number 'rev1'. then restore the incremental backups starting with 'rev1+1': cd /var/lib/codendi/backup/subversion/incr/projectname bunzip2 < projectname.incr.rev1+1:rev2.bz2 | svnadmin load /svnroot/projectname bunzip2 < projectname.incr.rev2+1:rev3.bz2 | svnadmin load /svnroot/projectname etc. • Last, restore appropriate permissions: chown -R codendiadm.projectname /svnroot/projectname Recover a Subversion repository Subversion repositories based on Berkley DB might get corrupted if a 'svn' command crashes. In that case, access to the Subversion repository is denied (even for read access, e.g. with viewVC). For instance, TortoiseSVN returns this error code: Error 500 - internal server error trying to update on project projectname In this case, you need to recover the repository. As root, run the following commands: svnadmin recover /svnroot/projectname chown -R codendiadm.projectname /svnroot/projectname In case the "svnadmin recover" command still fails you can use the low level repair utility of Berkeley DB: cd /svnroot/projectname/db db_recover -c -v chown -R codendiadm.projectname /svnroot/projectname Clean-up a Subversion repository Sometimes a user might ask you to clean-up a SVN repository because he made mistakes in the import for instance. Here is how to partially do it: Delete (or archive) the SVN repository: tar cfz /var/tmp/projname_svn.tgz /svnroot/projname rm -rf /svnroot/projname Then you need to clean-up the entries in the DB: • Launch phpMyAdmin
Slide 43: • • Get the repostory id from svn_repositories table. Manually execute this command DELETE FROM svn_commits WHERE repositoryid =your_repo_id; WARNING: This will remove visible entries, but will keep "zombie" entries in svn_checkins, svn_dirs and svn_files Set up a distributed architecture Introduction Codendi now supports a distributed architecture that allows some services (currently: Files and Subversion) to be located on satellite servers. Please note that this feature is still experimental. Setup In /etc/codendi/conf/local.inc: Choose a sys_server_id for the master and the satellite (any integer is OK) In /etc/codendi/conf/database.inc: both servers should point to the master MySQL server. In MySQL on the master server: Do not forget to enable remote access from the satellites in mysql : GRANT ALL ON codendi.* TO codendiadm@satellite.domain.com IDENTIFIED BY 'xxxx' Where 'xxxx' is the database password for codendiadm. See mysql documentation for details. In Codendi web interface: Create servers (in site administration panel / Servers Administration) according to your architecture. There should be one entry for each server (master or satellites). You must also select which server is the master on this page ('Choose the master'). In the service selection page of the creation process, select the appropriate server for the Subversion and File services. Currently, you need to be a site administrator to change the location of a service. Project configuration There are two ways to enable distributed services for a project: • • At creation time: select a template that has already set satellite services. In the service selection page, you will be able to choose the satellite for SVN and File services. Please note that the default template does not allow selection of satellite services. Once the project is created, it is still possible to set a satellite for one of the distributed services. However, only site administrators are able to set this parameter in the service administration page.
Slide 44: Migration It is possible to relocate a service that already has some historical data. Files Make a tarball of the /var/lib/codendi/ftp/codendi/projname directory on the server that contains the existing data, and un-tar the files at the same location on the new server. Subversion Similarly, make a tarball of the /svnroot/projname directory on the server that contains the existing data, and un-tar the files at the same location on the new server. Single Sign-on If you want Single Sign-on (SSO), all servers must be in a similar domain. Then, you must set the same cookie domain and the same cookie prefix for all servers (master and satellites). +------------------------------------+ | master.domain.com | +------------------------------------+ | sys_server_id =1 | | sys_cookie_prefix = 'CX1234' | | sys_cookie_domain = '.domain.com' | | sys_dbhost = 'master' | | | +-------------.----.=----------------+ _.-' `-._ .-' `-.._ _.-' `-._ .-' `-.__ _.-' +-------------.-'--------------------+ | sata.domain.com | +------------------------------------+ | sys_server_id =2 | | sys_cookie_prefix = 'CX1234' | | sys_cookie_domain = '.domain.com' | | sys_dbhost = 'master' | | | +------------------------------------+ `-._ +---------------------`-.------------+ | satb.domain.com | +------------------------------------+ | sys_server_id =3 | | sys_cookie_prefix = 'CX1234' | | sys_cookie_domain = '.domain.com' | | sys_dbhost = 'master' | | | +------------------------------------+ Note: If you use other Codendi servers in the same domain that are not part of the distributed architecture, they should use a different cookie prefix to avoid session collisions. Validators for users' password You can define rules to validate users' passwords. Here is an example of rules :
Slide 45: • • • • Password must contain at least 8 characters Password must contain at least 2 capital letter Password must contain at least 3 non-digit characters ... See site-content/*/account/password_strategy.txt for details. Add an expiration date on a user account As an administrator you can add an expiration date to a user account in several ways: • • • When creating a new user account, in the field expiration date. If you leave it blank, then no expiration date will be set. In the pending user interface, after user registration. By using the User Administration module, once you are on the user information page, you can add or modify the expiration date of a user account. Once the date is reached, the account status becomes suspended. If you want to expand the account validity, you have to reactivate the account and change the expiration date. How to change wiki language for a project Once the language of a wiki is set for a project, it is "impossible" to change it. If an admin made a mistake and activated the wiki for his project in the wrong language, it is however possible to change it. You will need to execute some SQL commands in PhpMyAdmin (for instance): 1. Search for the group_id of the project you want to re-init the wiki (let's call this group_id 'xxx'). 2. Execute the following SQL commands: • DELETE FROM wiki_attachment WHERE group_id = xxx • DELETE FROM wiki_attachment_log WHERE group_id = xxx • DELETE FROM wiki_group_list WHERE group_id = xxx • DELETE FROM wiki_log WHERE group_id = xxx • DELETE FROM wiki_page WHERE group_id = xxx 3. The wiki of the project has been removed. You can now activate it again with the right language. End of Document