2520
Comment: Initial notes, still incomplete this is a work in progress
|
12558
|
Deletions are marked like this. | Additions are marked like this. |
Line 1: | Line 1: |
`DRAFT - This is a work in progress` |
|
Line 6: | Line 4: |
== Are you ready? == Before you start make sure all bugs in the current active milestone on Launchpad are in the 'Fix Committed' state. It's also a good idea to review the Bazaar log to make sure the NEWS file contains information about all the important changes. When all the bug fixes and new features are ready in [[https://code.launchpad.net/~storm/storm/trunk|trunk]] it's time to create the release. == Preparing release-candidate packages == The ```python-storm``` package is made to be built with AutoPPA. You need to create a ```~/.autoppa.conf``` file with contents similar to: |
<<TableOfContents>> == Setting up the tools == The `python-storm` package is made to be built with AutoPPA. AutoPPA eases the process of building packages for all supported releases of Ubuntu. You need to create a `~/.autoppa.conf` file with contents similar to: |
Line 26: | Line 17: |
ppa = username branch = /home/username/src/projects/storm/trunk |
ppa = storm branch = /home/username/src/projects/storm/packaging |
Line 29: | Line 20: |
releases = dapper hardy intrepid jaunty karmic | releases = dapper hardy karmic lucid maverick |
Line 32: | Line 23: |
You'll also need something similar to this in your ```~/.dput.cf```: | The `repostory` directory must exist. The branch above should be a branch of `lp:~storm/storm/packaging`. It's best to `bzr checkout` the branch, to ensure that changes are uploaded back to the master copy. You'll also need something similar to this in your `~/.dput.cf`: |
Line 35: | Line 31: |
[username] | [storm] |
Line 37: | Line 33: |
method = ftp incoming = ~username/ppa/ubuntu/ |
method = sftp incoming = ~storm/ppa/ubuntu/ |
Line 43: | Line 39: |
Build the package in your own PPA so that you can test them before publishing them. Use the ```--no-merge``` option to build test packages: |
[[https://launchpad.net/autoppa|AutoPPA]] is available from Launchpad. ''Note: As of this writing, the PPA of autoppa is not up to date.'' You need to get the source of autoppa, and its dependency commandant, with a sourcecode checkout. |
Line 48: | Line 45: |
$ autoppa --no-merge storm-0.15-1~rc2 | bzr branch lp:commandant bzr branch lp:autoppa |
Line 51: | Line 49: |
``FIXME`` Provide instructions explaining AutoPPA installation. | Then you can include these packages with PYTHONPATH, by installing them directly in your system's Python, or by creating a virtualenv for the release process. == Are you ready? == 1. Make sure [[https://bugs.launchpad.net/storm|all bugs]] in the current active milestone on Launchpad are in the `Fix Committed` state. You may need to ask [[mailto:storm@lists.canonical.com|someone on the mailing list]] to create the next milestone so you can move the bugs that have not been closed to the next release. 1. Review the Bazaar log and make sure the NEWS file contains information about all the changes made since the last release. To do this, look at the bugs that have been closed for this milestone. Every closed bug should have a mention in the NEWS file. For each one that does not, add a description, gathered from some combination of the bug report, the branch commit messages, and the merge proposal. '''Be sure to include the Launchpad bug number in the NEWS entry!''' When all the bug fixes and new features are ready in [[https://code.launchpad.net/~storm/storm/trunk|trunk]] it's time to create the release. 1. Edit `storm/__init__.py` and set `version` to the new version. 1. Set the release date in the NEWS file. 1. Commit these changes 1. `bzr tag 0.XY` where `0.XY` is the version you are releasing 1. `bzr push :parent` 1. Merge the changes from `lp:storm` into `lp:~storm/storm/packaging`. == Preparing a release-candidate package == 1. Build the package in your own PPA so that you can test them before publishing them in the official Storm PPA: `autoppa build storm 0.18-1~storm1` ''Note: As of this writing, autoppa requires that gpg-agent be installed and running.'' {{{sudo apt-install gnupg-agent}}} will install it. Using {{{eval $(gpg-agent --daemon)}}} will get it running quickly, though you may want to make it run regularly for your sessions--see {{{man gpg-agent}}} if so. ''Note: The maverick build is failing for some developers as of this writing.'' This will cause autoppa to stop, failing to upload any of the packages to Launchpad. To work around this, remove {{{maverick}}} from the {{{releases}}} key of your {{{~/.autoppa.conf}}} file. ''Note: The PPA page on Launchpad can take a few minutes before reporting the newly-uploaded PPA.'' It may take a few hours to actually build. |
Line 54: | Line 97: |
specified in ```~/.autoppa.conf```. When the ```python-storm`` packages build you should test them in pristine environments, to make sure they work properly. If you run into problems you can build the package locally to debug issues: |
specified in `~/.autoppa.conf`. When the `python-storm` packages build you should test them in pristine environments, to make sure they work properly. Amazon's EC2 is good for this kind of testing. If you run into problems you can build the package locally to debug issues (run in the `packaging` branch): `fakeroot debian/rules binary` Any changes made for the build will have been committed to the `packaging` branch. == Preparing a release-candidate tarball == 1. Run `make release` to prepare a tarball (created in the `dist` directory). 1. This will create `dist/storm-0.17.tar.bz2`. Unpack it and make sure everything is in place: the README, NEWS, TODO and LICENSE files, the PKG-INFO and other distutils setup files, the storm and tests directories with Python code, the Makefile and the storm.egg-info directory. == Test the packages == Test the new packages. If everything looks good move onto the next step. Otherwise, fix the problems and build a new version of the package. Bump the `stormN` part of the version string to `stormN+1` and rebuild the package and tarball. == Creating and publishing the new tarball and packages == If the tarball and package are in order you can copy the package to the official Storm PPA and upload the tarball to Launchpad. 1. Find the new packages in your PPA and copy them to the Storm Team PPA to publish them. 1. Upload the tarball to Launchpad using the 'Create a release' function from the milestone page. Let Launchpad close the milestone as part of creating the release. Use the announcement contents to fill in the 'Release notes' and 'Detailed changelog' sections. 1. After the release is created use the 'Add file' function to upload the tarball. Be sure to also upload a valid GPG signature for it. Use 'Storm <version>' for the description of the file. You also should announce the package to PyPI. Run {{{python setup.py register}}} in the tree. == Build new documentation == 1. Run `make doc` in the `packaging` branch to build new API documentation. 1. Copy it to a public location and update the IRC topic. Make sure the link on the front page of the wiki is correct. Contact jkakar if you want to use the existing location and he will update the documentation for you. 1. Update the [[Tutorial]] to use the latest document from the source tree. == Announce the new release == 1. Make an announcement on Launchpad to publicise the release. Use the text 'Storm <version> is out!' for the headline and the text 'The latest stable release of Storm with new features and bug fixes is available.' for the summary. Set the URL to the milestone, such as: https://launchpad.net/storm/trunk/0.17 1. Change the topic on the IRC channel to reference the new version. 1. Write a nice release announcement for the Launchpad page and send a copy of it to the Storm mailing list to announce the release. The announcement from 0.16 is below and can be used as a template for new releases. In general, the announcement should (1) provide information about where to get the latest packages and tarball, (2) highlight the top 3-5 changes, including information about each one, and finally (3) the contents of the NEW file should be included. 1. Write a post on your blog to spread the word. |
Line 60: | Line 185: |
$ fakeroot debian/rules binary | The Storm team is proud to announce Storm 0.16! The new release includes a number of new features: * Storm's C extensions are enabled by default * Comparable expressions have new string comparison methods * The default object cache size has been changed * Set expressions use fewer stack frames during compilation * MySQL reserved words are correctly handled This release includes official packages for all supported releases of Ubuntu. They are available in the Storm team's PPA: https://edge.launchpad.net/~storm/+archive/ppa You can find the release files at: https://launchpad.net/storm/+download You can always get the latest source code from Launchpad: bzr branch lp:storm Finally, you can join us in the #storm channel on irc.freenode.net and on the Storm mailing list: https://lists.canonical.com/mailman/listinfo/storm Read on for more... Storm's C extensions are enabled by default ------------------------------------------- Storm has had C extensions for a long time, but until now they've been disabled by default. The extensions have a significant improvement on performance and are now used by default. The extensions can be disabled by defining a STORM_CEXTENSIONS=0 environment variable. When not available, Storm will automatically fall back to the equivalent Python versions of optimized code. Comparable expressions have new string comparison methods --------------------------------------------------------- Comparable expressions (such as Column and Alias) provide new startswith(), endswith() and contains_string() methods. These methods perform prefix, suffix and substring comparisons using LIKE. Strings used with these methods are automatically escaped. The default object cache size has been changed ---------------------------------------------- Storm 0.15 included an announcement about the default cache size being changed from 100 items to 1000 items. Due to a bug this information was actually incorrect. This is now fixed and should result in improved performance. Please report any problems or concerns about this change to the mailing list. Set expressions use fewer stack frames during compilation --------------------------------------------------------- The set expression constructor flattens the first argument, if it's part of the same type. The resulting expression tree uses fewer stack frames during compilation, reducing the chance of hitting Python's recursion limit. MySQL reserved words are correctly handled ------------------------------------------ In addition to the standard SQL92 reserved words, MySQL has it's own set reserved words. Storm is now aware of them and will perform escaping correctly, making it easier for users that have table and column names that conflict with this list. Detailed changelog ------------------ Improvements: - The set expression constructor will now flatten its first argument if it is of the same type. The resulting expression tree uses less stack when compiling so reduces the chance of hitting Python's recursion limit (bug #242813). - Add startswith(), endswith() and contains_string() methods to Comparable. These methods perform prefix, suffix and substring checks respectively using the LIKE operator, taking care of escaping for you (bug #387840). - C extensions are enabled by default. Define the STORM_CEXTENSIONS=0 environment variable to disable them (bug #410592). - The README file contains information about Storm's license and detailed instructions on setting up a development environment suitable for running the entire test suite. - 'make doc' uses Pydoctor to generate API documentation. - Integration tests for Django now work with Django 1.1. Bug fixes: - Remove a leak when mutable variables (ListVariable or PickleVariable instances) are collected before store.flush, leaving hooks behind them. - The ResultSet min, max and sum methods now work correctly when the result set is empty and the column has allow_none=False set. Previously this resulted in a NoneError (bug #457801). - MySQL reserved words are handled properly (bug #433833). - Test loading code has been simplified. Support for py.test has been removed in this process, as it was not functioning correctly and didn't fit into the PyUnit framework (bug #331905). - Remote diverged and remote deleted references now use a weak (Python) reference to the local object. This prevents a leak when the remote object stays in memory (bug #475148). - Check for invalidated state when returning the remote object of a relation: it fixes a bug if the local key of the Reference is the primary key (bug #435962). - The default Cache instance created for a Store honours Cache's default size. Store's docstring has been updated to reflect this (bug #374180). |
Line 63: | Line 307: |
If any changes had to be made to the package to get the new version to build they should be landed in a separate branch that is merged to trunk. == Preparing a release-candidate tarball == {{{ $ ./setup.py sdist --formats bztar }}} == Creating the final release packages and tarball == When the package is perfect and ready for redistribution rebuild it with the release version: {{{ $ autoppa storm-0.15-1~storm1 }}} This will upload sources, like before, but it will also create a branch with a revision storing the changes made for each release being built. This branch will be merged to trunk and a tag based on the package version will be added to the new revision to track it. |
== Prepare to develop the next version == 1. Mark all the bugs in the release milestone as 'Fix Released'. 1. If it hasn't already been done, create a milestone for the next version and target bugs that needs to be delivered with it. 1. Bump the `version` value in the `storm/__init__.py`. 1. Edit the `NEWS` file and add a new section for the development version. |
Preparing a release is a fairly straight-forward process despite appearances to the contrary.
Contents
Setting up the tools
The python-storm package is made to be built with AutoPPA. AutoPPA eases the process of building packages for all supported releases of Ubuntu. You need to create a ~/.autoppa.conf file with contents similar to:
[storm] email = Username <username@example.com> ppa = storm branch = /home/username/src/projects/storm/packaging repository = /home/username/src/autoppa-builds releases = dapper hardy karmic lucid maverick
The repostory directory must exist.
The branch above should be a branch of lp:~storm/storm/packaging. It's best to bzr checkout the branch, to ensure that changes are uploaded back to the master copy. You'll also need something similar to this in your ~/.dput.cf:
[storm] fqdn = ppa.launchpad.net method = sftp incoming = ~storm/ppa/ubuntu/ login = anonymous allow_unsigned_uploads = 0
AutoPPA is available from Launchpad.
Note: As of this writing, the PPA of autoppa is not up to date. You need to get the source of autoppa, and its dependency commandant, with a sourcecode checkout.
bzr branch lp:commandant bzr branch lp:autoppa
Then you can include these packages with PYTHONPATH, by installing them directly in your system's Python, or by creating a virtualenv for the release process.
Are you ready?
Make sure all bugs in the current active milestone on Launchpad are in the Fix Committed state. You may need to ask someone on the mailing list to create the next milestone so you can move the bugs that have not been closed to the next release.
- Review the Bazaar log and make sure the NEWS file contains information about all the changes made since the last release. To do this, look at the bugs that have been closed for this milestone. Every closed bug should have a mention in the NEWS file. For each one that does not,
add a description, gathered from some combination of the bug report, the branch commit messages, and the merge proposal. Be sure to include the Launchpad bug number in the NEWS entry!
When all the bug fixes and new features are ready in trunk it's time to create the release.
Edit storm/__init__.py and set version to the new version.
- Set the release date in the NEWS file.
- Commit these changes
bzr tag 0.XY where 0.XY is the version you are releasing
bzr push :parent
Merge the changes from lp:storm into lp:~storm/storm/packaging.
Preparing a release-candidate package
- Build the package in your own PPA so that you can test them before publishing them in the official Storm PPA:
autoppa build storm 0.18-1~storm1
Note: As of this writing, autoppa requires that gpg-agent be installed and running. sudo apt-install gnupg-agent will install it. Using eval $(gpg-agent --daemon) will get it running quickly, though you may want to make it run regularly for your sessions--see man gpg-agent if so.
Note: The maverick build is failing for some developers as of this writing. This will cause autoppa to stop, failing to upload any of the packages to Launchpad. To work around this, remove maverick from the releases key of your ~/.autoppa.conf file.
Note: The PPA page on Launchpad can take a few minutes before reporting the newly-uploaded PPA. It may take a few hours to actually build.
This will upload sources to your PPA for the Ubuntu releases specified in ~/.autoppa.conf. When the python-storm packages build you should test them in pristine environments, to make sure they work properly. Amazon's EC2 is good for this kind of testing. If you run into problems you can build the package locally to debug issues (run in the packaging branch): fakeroot debian/rules binary
Any changes made for the build will have been committed to the packaging branch.
Preparing a release-candidate tarball
Run make release to prepare a tarball (created in the dist directory).
This will create dist/storm-0.17.tar.bz2. Unpack it and make sure everything is in place: the README, NEWS, TODO and LICENSE files, the PKG-INFO and other distutils setup files, the storm and tests directories with Python code, the Makefile and the storm.egg-info directory.
Test the packages
Test the new packages. If everything looks good move onto the next step. Otherwise, fix the problems and build a new version of the package. Bump the stormN part of the version string to stormN+1 and rebuild the package and tarball.
Creating and publishing the new tarball and packages
If the tarball and package are in order you can copy the package to the official Storm PPA and upload the tarball to Launchpad.
- Find the new packages in your PPA and copy them to the Storm Team PPA to publish them.
- Upload the tarball to Launchpad using the 'Create a release' function from the milestone page. Let Launchpad close the milestone as part of creating the release. Use the announcement contents to fill in the 'Release notes' and 'Detailed changelog' sections.
- After the release is created use the 'Add file' function to upload the tarball. Be sure to also upload a valid GPG signature
for it. Use 'Storm <version>' for the description of the file.
You also should announce the package to PyPI. Run python setup.py register in the tree.
Build new documentation
Run make doc in the packaging branch to build new API documentation.
- Copy it to a public location and update the IRC topic. Make sure the link on the front page of the wiki is correct. Contact jkakar if you want to use the existing location and he will update the documentation for you.
Update the Tutorial to use the latest document from the source tree.
Announce the new release
- Make an announcement on Launchpad to publicise the release. Use
the text 'Storm <version> is out!' for the headline and the text 'The latest stable release of Storm with new features and bug fixes is available.' for the summary. Set the URL to the milestone, such as: https://launchpad.net/storm/trunk/0.17
- Change the topic on the IRC channel to reference the new version.
- Write a nice release announcement for the Launchpad page and send a copy of it to the Storm mailing list to announce the release. The announcement from 0.16 is below and can be used as a template for new releases. In general, the announcement should (1) provide information about where to get the latest packages and tarball, (2) highlight the top 3-5 changes, including information about each one, and finally (3) the contents of the NEW file should be included.
- Write a post on your blog to spread the word.
The Storm team is proud to announce Storm 0.16! The new release includes a number of new features: * Storm's C extensions are enabled by default * Comparable expressions have new string comparison methods * The default object cache size has been changed * Set expressions use fewer stack frames during compilation * MySQL reserved words are correctly handled This release includes official packages for all supported releases of Ubuntu. They are available in the Storm team's PPA: https://edge.launchpad.net/~storm/+archive/ppa You can find the release files at: https://launchpad.net/storm/+download You can always get the latest source code from Launchpad: bzr branch lp:storm Finally, you can join us in the #storm channel on irc.freenode.net and on the Storm mailing list: https://lists.canonical.com/mailman/listinfo/storm Read on for more... Storm's C extensions are enabled by default ------------------------------------------- Storm has had C extensions for a long time, but until now they've been disabled by default. The extensions have a significant improvement on performance and are now used by default. The extensions can be disabled by defining a STORM_CEXTENSIONS=0 environment variable. When not available, Storm will automatically fall back to the equivalent Python versions of optimized code. Comparable expressions have new string comparison methods --------------------------------------------------------- Comparable expressions (such as Column and Alias) provide new startswith(), endswith() and contains_string() methods. These methods perform prefix, suffix and substring comparisons using LIKE. Strings used with these methods are automatically escaped. The default object cache size has been changed ---------------------------------------------- Storm 0.15 included an announcement about the default cache size being changed from 100 items to 1000 items. Due to a bug this information was actually incorrect. This is now fixed and should result in improved performance. Please report any problems or concerns about this change to the mailing list. Set expressions use fewer stack frames during compilation --------------------------------------------------------- The set expression constructor flattens the first argument, if it's part of the same type. The resulting expression tree uses fewer stack frames during compilation, reducing the chance of hitting Python's recursion limit. MySQL reserved words are correctly handled ------------------------------------------ In addition to the standard SQL92 reserved words, MySQL has it's own set reserved words. Storm is now aware of them and will perform escaping correctly, making it easier for users that have table and column names that conflict with this list. Detailed changelog ------------------ Improvements: - The set expression constructor will now flatten its first argument if it is of the same type. The resulting expression tree uses less stack when compiling so reduces the chance of hitting Python's recursion limit (bug #242813). - Add startswith(), endswith() and contains_string() methods to Comparable. These methods perform prefix, suffix and substring checks respectively using the LIKE operator, taking care of escaping for you (bug #387840). - C extensions are enabled by default. Define the STORM_CEXTENSIONS=0 environment variable to disable them (bug #410592). - The README file contains information about Storm's license and detailed instructions on setting up a development environment suitable for running the entire test suite. - 'make doc' uses Pydoctor to generate API documentation. - Integration tests for Django now work with Django 1.1. Bug fixes: - Remove a leak when mutable variables (ListVariable or PickleVariable instances) are collected before store.flush, leaving hooks behind them. - The ResultSet min, max and sum methods now work correctly when the result set is empty and the column has allow_none=False set. Previously this resulted in a NoneError (bug #457801). - MySQL reserved words are handled properly (bug #433833). - Test loading code has been simplified. Support for py.test has been removed in this process, as it was not functioning correctly and didn't fit into the PyUnit framework (bug #331905). - Remote diverged and remote deleted references now use a weak (Python) reference to the local object. This prevents a leak when the remote object stays in memory (bug #475148). - Check for invalidated state when returning the remote object of a relation: it fixes a bug if the local key of the Reference is the primary key (bug #435962). - The default Cache instance created for a Store honours Cache's default size. Store's docstring has been updated to reflect this (bug #374180).
Prepare to develop the next version
- Mark all the bugs in the release milestone as 'Fix Released'.
- If it hasn't already been done, create a milestone for the next version and target bugs that needs to be delivered with it.
Bump the version value in the storm/__init__.py.
Edit the NEWS file and add a new section for the development version.