Help for working in migration 6.1 -> 7.0

Hi Sylvain,

thanks for your interest! I assume you already read http://openupgrade-server.readthedocs.org/en/latest/ ?

So the general idea is to provide migrations scripts for every addon in its migrations/[version] directory. This already happened for the base module in openupgrade-server's 7.0 branch. It has yet to be done for the addons in openupgrade-addons' 7.0 branch.

You will find a file named "openupgrade_analysis.txt" which helps you to decide what to do: Which fields are added, which ones removed, etc.

Then you can start coding: Do as much as you can in post-migration.py, resort to pre-migration.py when necessary. As the names indicate, the first is run after the migration, the second before. Before migration, you can assume the database to be unchanged (OpenERP has not yet created new fields and deleted old ones) but you have no access to the ORM, so you need to work directly on the database. After the migration, the ORM is available but old columns are dropped and field types changed for example. So it depends a lot on the module you're working on where the good place to do the work is. In your migration scripts, you can import openerp.openupgrade which provides some helper functions like renaming columns.

So to start working in a nutshell:
bzr branch lp:openupgrade-server
bzr branch lp:openupgrade-addons
[hack hack hack]
openupgrade-server/openerp/openerp-server --update=base,[the module(s) you worked on] --stop-after-init --database=[a copy of your 6.1 database] --addons-path=openupgrade-addons

It is crucial to point to openupgrade's addons path and not the original ones. This will take a while, you need to inspect your logfile closely and if everything went right, you have a database usable with 7.0 (of course, only the parts for which migrations scripts exist)

Please don't forget to share your results, propose a merge on openupgrade-addons then.

Hi

Thanks for your complete response. Yes I took a look in the readthedocs website. By the way, there is a page "module coverage 6.0 -> 6.1" but no page about "6.1 to 7.0".

First I have a question. The website & your response indicate two modules. (openupgrade-server for openobject-server and openupgrade-addons for openobject-addons). I use a third official module openerp-web. There is a openupgrade-web module too or it's not necessary ? (I don't know if my question is relevant...)

Another question I have : Is there "base" module complete ? for example in 6.1 there is no "logo_web" field in res.company and in 7.0 yes.
I saw nothing in openupgrade-server/openerp/addons/base/migrations files about the creation of that field.

Of course, i'll propose merge about my work.

+++

ad 0: Yes, this page has to be added. Currently it would only read 'base done, everything else tbd'. The docs are generated from http://bazaar.launchpad.net/~openupgrade-committers/openupgrade-server/7.0/files/head:/doc/, so if you while discovering the whole thing want to keep your findings there, it would be very much appreciated

ad 1: The web modules (up to now) don't mess with the database. They don't declare any models and given that changes in the models is all the openupgrade server cares about, we don't need to do anything with them

ad 2: Completely new fields are handled by the server already, you don't have to create them yourself.
You would have to do something for example if logo_web should contain some default derived from existing field(s)

Hi Sylvian,

Im also at the beginning of my path in implementing migration scripts for version 7.0 so Im also working on understanding the project and the opoenerp structure right now.

About the 'base' module: it has not been extensively tested for lack of other module migration scripts. In our experience, it will mature quickly with the first production migrations.

Also, it was written when the impact of the new partner/contact model was not clear to me yet. So do I expect some changes in the logic that handles the migration from addresses to contacts etc.

I want to use these scripts, I even watched a talk on how to use it on youtube. But I think I need some basics (and maybe too obvious) situations.

I did a bzr checkout but I see that scripts/ also have a whole openerp instance. So first question pops up:
- Do I need to use only the openerp from the openupgrade-server pull?

The documentation instructions on how to use openupgrade are a bit confusing regarding that:
"Edit the configuration files and command line parameters to point to the database you are going to upgrade. The parameters will probably be the same as you use on your live server, except they point to the OpenUpgrade addons source code, point to the database you want to upgrade, and add the –update all –stop-after-init flags. "

I would like to see some examples regarding where it needed to be addressed.

Hi.

Yes to "openupgrade" a database, you have to use the openupgrade project that is like a "fork" of openobject addons & server project with script in each addons and some script in openupgrade folders. (basically).
If you want to upgrade from version X to version Y, you have to use the Y version of openupgrade.
why there is the whole instance of openerp in openupgrade ?
-> that is usefull to have the whole framework when we write scripts migrations ;
-> openerp automatically add new fields and datas and remove obsolete fields and datas ; So the most important work is avoided.

after checking out openupgrade project, you have to run openupgrade like that :
python openupgrade_server_path/openerp-server --update=all --database=my_database -c my_config_file.conf

the --stop-after-init is not mandatory. if you don't use it, you can test the result by tiping http://localhost:8069.

To simply run a migration, you can use this script: http://bazaar.launchpad.net/~openupgrade-committers/openupgrade-server/7.0/view/head:/scripts/migrate.py that does all the magic

Thanks Holger, can you let me know the steps to make it work, should I just drop it on the legacy OE, the new OE, is there a config argument I should also include.

ATM I have something like this:
/home/user/public_html/openerp6/
/home/user/public_html/openerp7/
/home/user/public_html/openupgrade-server/
/home/user/public_html/openupgrade-addons/

I have the default database (sample database) with 6 common standard modules (crm, sales, purchase, projects, hr, accounting).

I download the script, I tried to run it as the following:
python migrate.py -C openerp6/install/openerp-server.conf --run-migrations="7.0" --branch-dir=/home/user/tmp/openupgrade

had to add the db_name=name to the openerp-server.conf on 6.x

As it ran it connect to the bzr repo and re-download the openerp-web/7.0

Hope I could have gone about this different, or is this good enough?

I ran the script and got the following errors on the stout:
http://pastebin.mozilla.org/3765418

OpenERP 7 seemed to have adopted the new database (it now has the one one on 6) but when I tried to login I got the following error:
http://pastebin.mozilla.org/3765405
OpenERP Server Error

Client Traceback (most recent call last):
  File "/home/jza/public_html/openerp7/openerp/addons/web/http.py", line 204, in dispatch
    response["result"] = method(self, **self.params)
  File "/home/jza/public_html/openerp7/openerp/addons/web/controllers/main.py", line 867, in authenticate
    req.session.authenticate(db, login, password, env)
  File "/home/jza/public_html/openerp7/openerp/addons/web/session.py", line 118, in authenticate
    if uid: self.get_context()
  File "/home/jza/public_html/openerp7/openerp/addons/web/session.py", line 170, in get_context
    self.context = self.model('res.users').context_get() or {}
  File "/home/jza/public_html/openerp7/openerp/addons/web/session.py", line 42, in proxy
    result = self.proxy.execute_kw(self.session._db, self.session._uid, self.session._password, self.model, method, args, kw)
  File "/home/jza/public_html/openerp7/openerp/addons/web/session.py", line 30, in proxy_method
    result = self.session.send(self.service_name, method, *args)
  File "/home/jza/public_html/openerp7/openerp/addons/web/session.py", line 103, in send
    raise xmlrpclib.Fault(openerp.tools.ustr(e), formatted_info)

Server Traceback (most recent call last):
  File "/home/jza/public_html/openerp7/openerp/addons/web/session.py", line 89, in send
    return openerp.netsvc.dispatch_rpc(service_name, method, args)
  File "/home/jza/public_html/openerp7/openerp/netsvc.py", line 292, in dispatch_rpc
    result = ExportService.getService(service_name).dispatch(method, params)
  File "/home/jza/public_html/openerp7/openerp/service/web_services.py", line 626, in dispatch
    res = fn(db, uid, *params)
  File "/home/jza/public_html/openerp7/openerp/osv/osv.py", line 188, in execute_kw
    return self.execute(db, uid, obj, method, *args, **kw or {})
  File "/home/jza/public_html/openerp7/openerp/osv/osv.py", line 131, in wrapper
    return f(self, dbname, *args, **kwargs)
  File "/home/jza/public_html/openerp7/openerp/osv/osv.py", line 197, in execute
    res = self.execute_cr(cr, uid, obj, method, *args, **kw)
  File "/home/jza/public_html/openerp7/openerp/osv/osv.py", line 185, in execute_cr
    return getattr(object, method)(cr, uid, *args, **kw)
  File "/home/jza/public_html/openerp7/openerp/tools/cache.py", line 18, in lookup
    r = self.lookup(self2, cr, *args)
  File "/home/jza/public_html/openerp7/openerp/tools/cache.py", line 46, in lookup
    value = d[key] = self.method(self2, cr, *args)
  File "/home/jza/public_html/openerp7/openerp/addons/base/res/res_users.py", line 363, in context_get
    res = getattr(user,k) or False
  File "/home/jza/public_html/openerp7/openerp/osv/orm.py", line 494, in __getattr__
    return self[name]
  File "/home/jza/public_html/openerp7/openerp/osv/orm.py", line 402, in __getitem__
    field_values = self._table.read(self._cr, self._uid, ids, field_names, context=self._context, load="_classic_write")
  File "/home/jza/public_html/openerp7/openerp/addons/base/res/res_users.py", line 810, in read
    res = super(users_view, self).read(cr, uid, ids, fields, context=context, load=load)
  File "/home/jza/public_html/openerp7/openerp/addons/base/res/res_users.py", line 272, in read
    result = super(res_users, self).read(cr, uid, ids, fields=fields, context=context, load=load)
  File "/home/jza/public_html/openerp7/openerp/osv/orm.py", line 3620, in read
    result = self._read_flat(cr, user, select, fields, context, load)
  File "/home/jza/public_html/openerp7/openerp/osv/orm.py", line 3672, in _read_flat
    cr.execute(query, [tuple(sub_ids)] + rule_params)
  File "/home/jza/public_html/openerp7/openerp/sql_db.py", line 161, in wrapper
    return f(self, *args, **kwargs)
  File "/home/jza/public_html/openerp7/openerp/sql_db.py", line 226, in execute
    res = self._obj.execute(query, params)
ProgrammingError: column res_users.alias_id does not exist
LINE 1: SELECT res_users."menu_id",res_users."alias_id",res_users."a...

Hi Alexandro,

please post your own question instead of hijacking this thread. I will post my ideas about your problem there.

Cheers,
Stefan.

Sorry Stefan,
done, +question/240731