Table of Contents
- Table of Contents
- What is L10n-stats ?
- L10n-stats Configuration
- Web Translation Statistics Configuration
- SQL Schema Description
- Required tree of localization repository
- Use a newer version
- Request and Bug reports
- Todo list
The goal of this project is to provide an easy preview of the localization status of a project. Generally, a project is divided in different parts, to be able to be translated by several people at the same time, in different languages.
These parts are called PO files, and these files are analyzed by l10n-stats, to display translation status of each one, for each language supported.
Of course, a web site demo will explain better than thousand words the use you can have of the l10n-stats project. Please see, for example, the KDE localization web site http://l10n.kde.org/
You should find a more updated documentation here.
The scripts use the PHP language, and will require the PHP command line to work.
What is l10n-stats ?
L10n-stats is a set of PHP scripts used with a MySQL database providing HTML pages of the localization status of a project using Gettext. This project was written initially for KDE project, and a demonstration could be found at http://l10n.kde.org
The l10n-stats scripts regurlarly update a database to see how your project is translated, by translation teams, by packages, and even by specifical branches.
It uses a local copy of any SVN repository to process PO files, saves these information to a MySQL database, and then datas are processed using PHP scripts.
The only requirement you need is to have previously organized your localization tree files like the KDE project. We will talk about this in an other paragraph. Do not be afraid, this will not be a problem if your project is correctly organized.
Download a tarball of the current version of l10n-stats from the download page of this project. Uncompress it in your home folder for example, or whatever.
Then, please copy the configuration file (l10n-stats.conf) to your home directory root and rename it to .l10n-stats.conf (do not forget the first "." in the name) You can alternatively copy this file to the /etc folder : /etc/l10n-stats.conf.
For more information, you can download the default config file here : l10n-stats.conf to see how it looks like.
The parameters of the .l10n-stats.conf file in your home folder will be used by default by l10n-stats. The l10n-stats.conf file in the l10n-stats folder just contains default parameters, and they will be overriden by the one you set up in .l10n-stats.conf.
A good solution is to only keep configuration entries that you have modified in your .l10n-stats.conf file, because most of configuration entry are for advanced users, and do not need to be changed.
- Create a database in your MySQL server, with the name you want. Configure the .l10n-stats.conf to allow scripts to connection to this database correctly.
- Use the tables-creation.sql SQL scripts to create needed tables of l10n-stats. (through mysql command line, or phpMyAdmin if installed.)
- Create a working copy of your SVN repository on the machine you're going to use l10n-stats scripts. This working copy must respect some specifical criteria, described in an other paragraph.
- Use the datas-examples.sql SQL example to configure properly your project and then execute this script on your MySQL database.
Configure the different options of the .l10n-stats.conf file.
Generally, you only need to modify paths to SVN repository and login/password to access to your database.
Test your installation by launching the following script of l10n-stats project : stats-datas.script.php
Grabbing Statistics (stats-datas.script.php)
Read local working copy of a SVN repository which contains PO files of a specifical project. See the KDE SVN project to know how to order your repository (http://websvn.kde.org) The statistics are saved in a MySQL database previously created. This first task is mandatory if you want to generate pages. To see graphical history, use the Crontab to launch this script everyday.
You can use the following lines to regurlarly update and regenerate your statistics page.
Use the crontab -e command and type the following lines :
############ L10n pages generation ############## # Launch stats-datas.script.php script, everyday, at 4h o'clock 00 4 * * * cd
/ ; ./stats-datas.script.php
Web Translation Statistics Configuration
The PHP scripts displaying statistics pages are available here : http://websvn.kde.org/trunk/www/areas/l10n/stats
Simply checkout these scripts in the stats/ folder of your website. You will also need an .htaccess file and the media/ framework of KDE website.
So, simply execute the following commands :
mkdir -p /var/www/statistics cd /var/www/statistics svn export svn://anonsvn.kde.org/home/kde/trunk/www/areas/l10n/stats svn export svn://anonsvn.kde.org/home/kde/trunk/www/areas/l10n/img svn export svn://anonsvn.kde.org/home/kde/trunk/www/areas/l10n/menu.inc svn export svn://anonsvn.kde.org/home/kde/trunk/www/areas/l10n/site.inc svn export svn://anonsvn.kde.org/home/kde/trunk/www/areas/l10n/css.inc svn export svn://anonsvn.kde.org/home/kde/trunk/www/areas/l10n/.htaccess svn export svn://anonsvn.kde.org/home/kde/trunk/www/media
Then you need to copy the l10n.kde.org.conf Example File to the following path of your server :
/etc/l10n.kde.org.confOf course, you need to configure login and password to access to your database in this configuration file.
Apache Virtual Host Configuration
Please note that Web Translation Statistics needs an enabled mod_rewrite in your Apache 2.
Here is an example of a working Virtual Host configuration :
<Directory "/var/www/statistics"> Options -Indexes +FollowSymLinks AllowOverride All Order allow,deny Allow from all </Directory> <VirtualHost *:80> DocumentRoot "/var/www/statistics" ServerName statistics.com ServerAdmin email@example.com Alias /media /var/www/media php_value include_path ".:/var/www/media/includes:/usr/share/php5" ErrorLog /var/log/apache2/statistics-error.log TransferLog /var/log/apache2/statistics-access.log </VirtualHost>
- When everything is ready (do not forget to restart your Apache if you have enabled new modules or sites), you can try to consult the URL :
- If all seems to work, and if PHP scripts are correctly displayed in your browser, then you have finished to configure l10n-stats.
Customize Web Translation Statistics Style
If you would like to use a different style from KDE for your website, here is a little description of main files :
- This folder contains the template layouts of pages header and footer.
- Default CSS stylesheet for KDE. The modification of these CSS will help you to customize the general web site layout (header, footer, menu,...)
- Default images for KDE website. Add your own images in this folder to customize your website.
- This stylesheets defines the statistics table style and appearance. Its modification is particularly useful if you would like to change the layout of the statistics pages.
For more information do not hesitate to consult the media/ framework documentation.
SQL Schema DescriptionYou can download the SQL script example to see what you need to configure : datas-examples.sql. The SQL script in charge of creating tables is available here : tables-creation.sql.
This table contains branches (aka "revision") of your project. The root folder containing the working copy of your repository must contain each branches in one folder. In our example, these revisions are trunk and stable.
The second value allow the administrator to disabled the wanted revisions, using the following values : 1 for an enabled revision, and 0 for a disabled one.
The third value let you select the revision as main revision of your project. Only one revision could have this value at 1.
Tip : If your repository does not look like this, you can use multiple svn checkout command to create a working copy that looks like the required one. See your SVN manual for more information.
INSERT INTO `l10n_revisions` VALUES ('trunk',1,1); INSERT INTO `l10n_revisions` VALUES ('stable',1,0);
This table contains all teams of your localization project. The first value is the team code (and the name of the folder where PO files can be found). The second value is for displaying the name of the team.
INSERT INTO `l10n_teams` VALUES ('af','Afrikaans'); INSERT INTO `l10n_teams` VALUES ('ar','Arabic'); INSERT INTO `l10n_teams` VALUES ('az','Azerbaijani'); INSERT INTO `l10n_teams` VALUES ('be','Belarusian');
This table contains the different packages of your project for GUI (in fact, these packages represents the different folders of your repository tree).
This allows you to group PO files of your localization project easily
INSERT INTO `l10n_guicfgpackage` VALUES ('kdeaddons'); INSERT INTO `l10n_guicfgpackage` VALUES ('kdeadmin'); INSERT INTO `l10n_guicfgpackage` VALUES ('kdeartwork'); INSERT INTO `l10n_guicfgpackage` VALUES ('kdebase');
This table contains files and packages which are considered as essential for your project.
It means that these files must be translated in priority by new localization teams. It refers in fact to the Essential pages of the generated web site.
The 'f' signify an essential file, and the 'p' an essential package folder. The second value is the name of the file/package, and finally, the last value is the minimal translation state required for considering localization nearly ready to use.
INSERT INTO `l10n_guicfgessential` VALUES ('f','desktop_l10n.po',75); INSERT INTO `l10n_guicfgessential` VALUES ('f','desktop_kdelibs.po',75); INSERT INTO `l10n_guicfgessential` VALUES ('f','kdelibs.po',90); INSERT INTO `l10n_guicfgessential` VALUES ('p','kdebase',75); INSERT INTO `l10n_guicfgessential` VALUES ('f','desktop_l10n.po',75); INSERT INTO `l10n_guicfgessential` VALUES ('f','desktop_kdelibs.po',75); INSERT INTO `l10n_guicfgessential` VALUES ('f','kdelibs.po',90); INSERT INTO `l10n_guicfgessential` VALUES ('p','kdebase',75);
Tables for Documentation statistics
For documentation generation configuration SQL tables, use the same templates SQL as above, and simply replace the l10n_gui prefix by this one l10n_doc. The options are identical to the one described in GUI generation.
Tip : You can also, if you want, use same tables by reconfiguring tables name in your config file, too have same tables for Doc and Gui profiles.
Required tree of localization repository
As you can see in the previous paragraph, you need to represent exactly what your project looks like. This helps l10n-stats to know how to generate web site tree, but also where to find PO files in your working copy.
To sum up, here is the kind of working copy l10n-stats projects will look for :
The scripts need to find the following working copy tree :
SQL tables(field name) are marked in bold, and config entry in italic. Other things are static to every working copy.
Example for GUI PO files retrieving :
/home/stats/ trunk/l10n/ fr/messages package1 package2 en/messages package1 package2 stable/l10n/ fr/messages package1 package2 en/messages package1 package2
Example for Documentation PO files retrieving :
/home/stats/ trunk/l10n/ fr/docmessages package1 package2 en/docmessages package1 package2 stable/l10n/ fr/docmessages package1 package2 en/docmessages package1 package2
If your current SVN repository do not respect this architecture, you can easily build a similarly one using multiple svn checkout, for example, for each revisions, and language. Please however note that this architecture seems to be quite powerful because it is used by KDE, which is a very big project.
For more details, see the KDE SVN repository, on which l10n-stats is based.
My scripts could not be launched, the following error appears (or a similar one) :
ERROR Documentation statistics stats-datas.script.php (): : status=error started=3/3/2006 23:29:0 mesg=There's another stats-datas.script.php instance running. Aborting...
This error generally occurs when you try to launch at the same time the same script. It can also occur if the previous execution failed. In this case, you need to remove the lock file /tmp/stats-datas.script.php.lock.
Use a newer version
If you want to use a more recent version of l10n-stats, it should be really easy if you have correctly followed this documentation.
First, copy your template folder to another place and be sure the config file is still in your home (.l10n-stats.conf). Once these tasks are done, simply remove old l10n-stats folder, download and uncompress the new version, and recopy template folder to this new version. Just watch new parameters and documentation about new features and test it. That's all !
Contact Nicolas Ternisien <firstname.lastname@example.org> if you have any questions or requests about l10n-stats project. Nicolas Ternisien is the current maintainer of l10n-stats and the l10n.kde.org web site.
You can still visit the official web site of l10n-stats project : http://l10n.kde.org/about-stats.php
The original author of these scripts is Claudiu Costin.
The following people made nice suggestions for l10n-stats project, about GUI statistics generation (alphabetical order) :
- Dimitrios Kamenopoulos <email@example.com>
- Hunted many spelling and grammatically mistakes I've made.
- Dwayne Bailey <firstname.lastname@example.org>
- Requested the history graphics to see overall progress of translation process per team and per project.
- Eric Bischoff <email@example.com>
- HTML visual appearence suggestions, bugs reporting, found spelling mistakes.
- Funda Wang <firstname.lastname@example.org>
- Spotted little bug on navigation page on second level; suggested to add usefull links in partial translation page for second level. Also observed the flawled math effect on graph bars.
- Konrad Dabrowski <email@example.com>
- Sent me very detailed report about math errors I've made. These errors have been very tricky because showed up on limited combination of statistics numbers.
- Krzysztof Lichota <firstname.lastname@example.org>
- Reported a new math bug caused by little typos: fuzzy percentage in lines and totals always have been multipled by 2. Spotted a missing CVS branch variable in cvsweb URL for PO files on 3rd level statistics files.
- Pedro Morais <email@example.com>
- Found display bug in "Top 10", suggested the "Partial Translations" page with all features it has.
- Thomas Diehl <firstname.lastname@example.org>
- Lots of suggestions for GUI Statistics content, page format, structure and many other hints.
The following people made nice suggestions for l10n-stats project, about Documentation generation (alphabetical order) :
- Amilcar do Carmo Lucas <email@example.com>
- Observed that "About" and "Help" pages are missing in KDE DOC Statistics.
- Erik K. Pedersen <firstname.lastname@example.org>
- Reported that PO files in kdelibs wouldn't be taken in account when grabbing statistics.
- Pedro Morais <email@example.com>
- Pushed me ahead when I was lazy. Promptly pointed me to configure and fix stats when KDE switched to new releases.
- Federico Zenith <firstname.lastname@example.org>
- Sugested better explanation for "update" icon.