/program/lib/updatelib.php - update wizard
This file handles all system updates. The basic idea is as follows.
We assume that a previous version of Website@School is/was already correctly installed using the code in /program/install.php. Using this version the school has added many, many hours of work in entering data.
Now a new version is installed, i.e. the new files (or just the updated files) are copied/uploaded to the webserver, including the file version.php and perhaps also updated versions of modules, themes, etc. This yields an error message for visitors (the infamous 'error 050') because the database version and the file version no longer match. The user logging in into admin.php is forced to attend to the job 'update', arriving here in job_update().
There an overview is presented of the core version and versions of all subsystems, with the option to upgrade those that qualify. The actual work for the core version is done in update_core() in this file. The actual work for modules and themes are done via the code in those subsystems. However, these are called from here.
The user is more or less _forced_ to perform the upgrade: all ways lead to job_upgrade() while the internal (database) version does not match the file version.
The upgrade should perform all necessary steps to upgrade, ending with updating the internal version number to match the file version. After that the error message '050' for visitors is gone, and admin.php no longer forces the user to come here. It is possible, however, to manually arrive here to check the updates of modules, themes, etc. but I consider that less important. That is: an upgrade of a module or theme will NOT be forced upon the user. It is wise, though to upgrade, but that is up to the user.
retrieve an array of manifests for modules, themes or languages
this examines the file system starting in the directory $path, looking for manifest files. These manifest files are named after the subdirectory they are in as follows. Example: If $path is /program/modules, this routine steps through that directory and may find subdirectories 'htmlpage', 'guestbook' and 'forum'. Eventually these manfest files are include()'d: /program/modules/htmlpage/htmlpage_manifest.php, /program/modules/guestbook/guestbook_manifest.php and /program/modules/forum/forum_manifest.php.
Every manifest file must describe the module (or language or theme) via the following construct:
After processing all the subdirectories of $path, the resulting array $manifests is returned. Note that pseudo-directories like '.' and '..' are not considered. Also, subdirectories 'foo' without the file 'foo_manifest.php' are also ignored.
Note that the name of the manifest file itself is also stored in the array, but excluding the subdirectory name.
Note: a similar routine is used in the installation script install.php.
install an additional language pack
this routine attempts to insert the information from the manifest of language $language_key into the database. The routine displays the result (error or success) in a message in $output. Details can be found in the logs.
The language_key is validated by reading all existing manifests. This is quite expensive, but that is not important because we do not use this routine very often anyway.
Note that we assume that the actual translations of the language pack are already unpacked into the correct directories. The corresponding manifest should exist in the directory /program/languages/$language_key.
install an additional module
this routine attempts to insert the information from the manifest of module $module_key into the database. The routine displays the result (error or success) in a message in $output. Details can be found in the logs.
The module_key is validated by reading all existing module manifests. This is quite expensive, but that is not important because we do not use this routine very often anyway.
Note that we assume that the actual modules are already unpacked into the correct directories. The corresponding manifest should exist in the directory /program/modules/$module_key.
install an additional theme
this routine attempts to insert the information from the manifest of theme $theme_key into the database. The routine displays the result (error or success) in a message in $output. Details can be found in the logs.
The theme_key is validated by reading all existing module manifests. This is quite expensive, but that is not important because we do not use this routine very often anyway.
Note that we assume that the actual thenes are already unpacked into the correct directories. The corresponding manifest should exist in the directory /program/themes/$theme_key.
main entry point for update wizard (called from /program/main_admin.php)
This routine takes care of executing update routines for both the core program and modules, themes, etc. It is called automagically whenever the core program version in the database is different from the version in the file {@lnk version.php} (see also main_admin()).
It can also be called manually via 'job=update'. When no specific task is specified, this routine shows the overview of versions for core, modules, themes, etc. Whenever a component is NOT up to date, an [Update] button is displayed. If a component IS up to date, we simply display the word 'OK'. This implies that when everything is up to date, the overview simply displays a list of OK's and the user is 'free to go'.
The actual updates for modules, themes, etc. is done via the various subsystems themselves, e.g. by calling htmlpage_upgrade() in the file /program/modules/htmlpage/htmlpage_install.php. The updates for the core program are actually performed from this file right here, see below for an example.
Note that we give a core update high priority: if the core is not up to date, nothing will work, except updating the core.
update the core version in the database to the version in the version.php file (the 'manifest' version)
perform actual update to version 2010120800
perform actual update to version 2010122100
perform actual update to version 2011020100
perform actual update to version 2011051100
this is a substantial change in the database: we (finally) standardise on UTF-8 including the database. Up until now we still only have a choice of exactly one database driver: MySQL. Therefore the upgrade we do here can be more or less MySQL-specific. (So much for database-independency).
What needs to be done here?
The most important task (in fact: the only task) is to change the collation (and implicitly the default charset) to utf8_unicode_ci (4.1.x <= MySQL < 5.5.2) or utf8mb4_unicode_ci (MySQL 5.5.3+). See mysql.class.php for more information on these UTF-8 & MySQL issues.
Strategy here is as follows.
This way we _might_ be able to work our way through huge tables: if the PHP max processing time kicks in, we can rerun the upgrade and start (again) with the table we had in our hands the previous time. I don't expect this to happen, but it still the way to do it IMHO.
Note that I assume that I cannot change the default charset of the DATABASE for the same reason the Installation Wizard expects the database to be ready before installation commences. (I cannot be sure that I have the privilege to execute 'ALTER DATABASE $db_name DEFAULT CHARSET utf8 COLLATE utf8_unicode_ci').
A useful reference for solving this problem of converting to utf8 can be found here: http://codex.wordpress.org/Converting_Database_Character_Sets.
In the case of W@S we do not have to deal with enum-fields because those are not used at this time. In fact it boils down to changing char, varchar, text and longtext.
Let goforit...
perform actual update to version 2011093000
this is yet another substantial change in the database: after we (finally) standardised on UTF-8 the last time (see update_core_2011051100() a number of problems occurred with new installations.
This specifically occurs with MySQL (currently the only supported database). In all their wisdom Oracle decided to change the default database engine from MyISAM to InnoDB in MySQL version 5.5.5. Bad move to do that somewhere in a sub-sub-release. Anyway. New installations with the default InnoDB engine AND with the 4-byte utf8mb4 character set (available since sub-sub-release 5.5.3) now generate serious trouble, because
These two conditions (InnoDB and utf8mb4) limit the length of a key (index) to 767 bytes / 4 bytes-per-char = 191 utf8mb4 characters. As it happens, some tables in WebsiteAtSchool used keyfields like varchar(240) and even varchar(255). These key sizes fail in InnoDB/utf8mb4 and the latter even fails with MyISAM/utf8mb4 because 255 * 4 + 2 = 1022 bytes > 1000 bytes. What a mess...
So there you have it: all keys MUST be shortened to 191 characters max. in order to prevent stupid error messages about key too long. The alternative (forcing another character set such as 'ascii' or 'latin1' for some fields) doesn't cut it IMHO.
*sigh*
We still have a choice of exactly one database driver: MySQL. Therefore the upgrade we do here can be more or less MySQL-specific (so much for database-independency), as it has to be, because the syntax of ALTER TABLE is -- unsuprisingly -- MySQL-specific.
The good news is that we are still in beta, so a major change in the data definition is less painful than with hundreds of production servers...
Another issue is the use of foreign keys. We used to have a FK in the nodes tabledef along the lines of this construct: FOREIGN KEY parentnode (parent_id) REFERENCES nodes (node_id); Upto now this could not possibly have worked with InnoDB because adding a node would at the top level of an area would not satisfy this constraint. Since MyISAM silently ignores any foreign key definition it 'simply works' in that case. So, because this FK must be removed from earlier installations we need to DROP the FOREIGN KEY. However, since the whole program never installed using InnoDB, there is no need to drop this foreign key that wasn't even recorded (in a MyISAM database) in the first place. The same applies to a number of other FK's too: these are now removed from the various tabledefs but do no need to be DROPped in this update routine.
What needs to be done here?
For existing tables some fields must be shortened from varchar(255) or varchar(240) to something like varchar(191) or even less. This MUST be done for key (index) fields. However, while we are at it some more fields SHOULD (or COULD) be shortened too. Here is what we do.
Below is a discussion of all affected fields and the rationale for picking the new lengths less than 191 characters.
Currently the longest parameter name in use is 27 characters, so I have to admit that the arbitrary size of 240 is a little bit too much. I'll reduce these fields to a size of 80, which seems a little more realistic. As an additional bonus, this allows for a compound key using 'section' and 'name' in users_properties while staying within the limit of 767 bytes or 191 characters.
and
The length of username or groupname was arbitrary set to 255. Different systems have different limits, e.g. 8, 14, 15, 16, 20, 32, 64 or 128. Since W@S is a stand-alone system we are more or less free to choose whatever we want (as long as it is less than 191 of course).
Since a username or groupname is only used to distinguish one user from another but at the same time giving at least some readability, a length of 255 is way too long. An arbitrary but hopefully more realistic choice is 60 characters.
The path for a user or group is derived from the corresponding name so it makes sense to make both fields the same length.
A remote address of type IPv4 generally looks like this: 'ddd.ddd.ddd.ddd' => length 15 It is not so easy to determine the length of an IPv6 address, because many valid variants exist. 'xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx' => length 39 '0000:0000:0000:0000:0000:0000:ddd.ddd.ddd.ddd' => length 45 '[0000:0000:0000:0000:0000:0000:ddd.ddd.ddd.ddd'] => length 47 (RFC3989)
Adding to the complexity and confusion are link-local addresses with zone indices: a percent-sign followed by an interface number (e.g. '%1') or interface name (e.g. '%eth0') appended to the raw address. This adds 2 or 5 or even more characters to the address. And then we of course have the reverse DNS-variant like 'x.x.x.x.x.x.x.x.x.x.x.x.x.x.x.x.x.x.x.x.x.x.x.x.x.x.x.x.x.x.x.x.ip6.arpa.' => length 73 or the special Microsoft trick to shoehorn a literal address in a UNC path: 'xxxx-xxxx-xxxx-xxxx-xxxx-xxxx-xxxx-xxxx.ipv6-literal.net' => length 56 or 'xxxx-xxxx-xxxx-xxxx-xxxx-xxxx-xxxx-xxxxs1.ipv6-literal.net' => length 58+ (with zone index)
Of course there several 'simplifications' such as omitting leading zeros in the hexquads and replacing the longest sequece of 0-hexquads with '::' that add to the confusion. RFC5952 adds the definition of a 'canonical representation' of IPv6 addresses to the party. Mmmm, see http://xkcd.com/927
My conclusion is: this whole IPv6-idea suffers from the Second System Syndrome (see F. Brooks' Mythical Man Month) and unfortunately we have to deal with it.
*sigh*
I will reduce the length of these fields from 255 to 150 for no other reason than that it is 10 times the length of a dotted-decimal IPv4 address and sufficient to accomodate a reverse DNS address twice (2 x 73 = 146).
This field stores a session key, currently constructed using md5() which yields a string with 32 (lowercase) hexadecimal characters. In the future a different digest could be used to provice a session_key, e.g. SHA-1 (40 hexdigits) or SHA-512 (128 hexdigits). Another option would be to use a UUID: 128 bits represented in 32 hexdigits in the form xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx (string of 36 bytes). Alternatively, the SHA-512 could be encoded in base64 yielding a string of 512 / 6 = 86 bytes. In this context, a field of size 255 seems a little over the top, not to mention problematic with 4-byte UTF-8 characters combined with the infamous MySQL / InnoDB-limit of 767 bytes for keyfields. I guess I will settle for a field size of 172 characters which is not too much for InnoDB keys + utf8mb4 and exactly enough to store a 1024 bit number in base64.
perform actual update to version 2012041900
Changes between 2011093000 and 2012041900:
perform actual update to version 2013071100
Changes between 2012041900 and 2013071100:
perform actual update to version 2014111700
Changes between 2013071100 and 2014111700:
perform actual update to version 2016062900
Changes between 2013071100 and 2016062900:
record the specified version number in the config table AND in $CFG->version
This utility routine records the new version number in the config table and also adjusts the version number already in core (in $CFG->version).
create table in database from an individual tabledef
create tables in database via include()'ing a file with tabledefs
update a language in the database
this routine tries to update the information in the database with the information in the language manifest of the selected language $language_key. The event is logged via logger().
Note that an upgrade of a language is not at all interesting because there is nothing to do except to update the data in the databse with that from the manifest. However, we still do it this way in order for the user to grow accustomed to it so we can complexicate this routine in the future without the user having to learn new tricks.
call the module-specific upgrade routine
this routine tries to execute the correct upgrade script/function for module $module_key. If all goes well, a success message is written to $output (and the update is performed), otherwise an error message is written to $output Either way the event is logged via logger().
Note that we take care not to load spurious files and execute non-existing functions. However, at some point we do have to have some trust in the file system...
attempt to remove or at least flag obsolete files
this routine can grow bigger on every update when perhaps more files are obsoleted. We always check all files (even the older ones) because the user might not have removed them yet. If we can delete the files, we do so. If not, we log it and also show a message to the user via $output.
display an introductory text for update + status overview
return an anchor tag with link to the specific update function
This utility routine returns a ready to user HTML anchor tag.
close the status overview HTML-table we opened before
this is the companion routine for update_status_table_open(); it closes the open HTML-table
open a status overview HTML-table including column headers
this routine opens an HTML-table in prepration for a status overview of the system or a subsystem (languages, modules, themes). The optional title is used as the header of the first column.
The width of the first column is 25% and the remaining 5 columns area 15% each which creates an orderly display of name, internal version, external version, releasedate, release and status.
call the theme-specific upgrade routine
this routine tries to execute the correct upgrade script/function for theme $theme_id. If all goes well, a success message is written to $output (and the update is performed), otherwise an error message is written to $output Either way the event is logged via logger().
Note that we take care not to load spurious files and execute non-existing functions. However, at some point we do have to have some trust in the file system...
Documentation generated on Tue, 28 Jun 2016 19:12:25 +0200 by phpDocumentor 1.4.0