Zodbupdate transaction error

Asked by mallow

I am attempting to run zodbupdate on the Data.fs from the Silva 2.1 site in order to upgrade to Silva 2.2 but I get this error when I run zodbupdate, any thoughts?

[AUser@development silva-2.2]$ ./bin/zodbupdate -f /svn/silva-2.2/var/filestorage/Data.fs
Ignoring index for /svn/silva-2.2/var/filestorage/Data.fs
/svn/silva-2.2/var/filestorage/Data.fs data record exceeds transaction record at 1480146987
Traceback (most recent call last):
  File "./bin/zodbupdate", line 123, in ?
    zodbupdate.main.main()
  File "build/bdist.linux-i686/egg/zodbupdate/main.py", line 73, in main
  File "/svn/silva-2.2/parts/zope2/lib/python/ZODB/FileStorage/FileStorage.py", line 165, in __init__
    read_only=read_only,
  File "/svn/silva-2.2/parts/zope2/lib/python/ZODB/FileStorage/FileStorage.py", line 1650, in read_index
    name, pos)
  File "/svn/silva-2.2/parts/zope2/lib/python/ZODB/FileStorage/FileStorage.py", line 53, in panic
    raise CorruptedTransactionError(message)
ZODB.FileStorage.FileStorage.CorruptedTransactionError: %s data record exceeds transaction record at %s

Question information

Language:
English Edit question
Status:
Answered
For:
Silva Edit question
Assignee:
No assignee Edit question
Last query:
Last reply:
Revision history for this message
Sylvain Viollon (thefunny) said :
#1

It sounds like your Index is corrumpted. I recommand you to delete the Data.fs.index file, only the .index file. It will be rebuilt the next time you open the database (like re-run zodbupdate), it can take few minutes if your database is big.

I never saw this error before, but from this I doubt that you Zope will start in any case with this database.

Are you sure to have properly copy your database (and completely) ?

Revision history for this message
mallow (princemallow) said :
#2

You are right the Data.fs was corrupt, the md5 of the repozo backup I made on the actual server did not match the md5 of the file after it was downloaded, probably because it was so big. So instead I used PSCP? (a putty extention) to download it and after a few hours the md5 matched. After I restore the backup with repozo I am able to go through the ZODBupdate process however I run into a new roadblock.

The page that shows up after pressing the Upgrade Content on the silvaroot-url/service_extensions/manage_extensions shows an error:

System problem
Something went wrong. Please use your back button to return to the previous page. Sorry for the irritation.
Error Type: ComponentLookupError
Error Value: (, '')

[Here is the output of the zodbupdate from the console]

$ ./bin/zodbupdate -f ./var/filestorage/Data.fs
Loaded 5 rules from silva.core.upgrade:renames
New implicit rule detected BTrees._OOBTree OOBTree to BTrees.OOBTree OOBTree
New implicit rule detected BTrees._IOBTree IOBTree to BTrees.IOBTree IOBTree
New implicit rule detected BTrees._OIBTree OIBTree to BTrees.OIBTree OIBTree
New implicit rule detected BTrees._IOBTree IOBucket to BTrees.IOBTree IOBucket
New implicit rule detected BTrees._IIBTree IIBTree to BTrees.IIBTree IIBTree
New implicit rule detected BTrees._OIBTree OIBucket to BTrees.OIBTree OIBucket
New implicit rule detected BTrees._IIBTree IITreeSet to BTrees.IIBTree IITreeSet
New implicit rule detected BTrees._IIBTree IISet to BTrees.IIBTree IISet
Warning: Missing factory for Products.Annotations.AnnotationTool AnnotationTool
/svn/silva-2.2/eggs/Products.SilvaFind-1.2-py2.4.egg/Products/SilvaFind/__init__.py:6: DeprecationWarning: registerDirectory is deprecated. Please import directly from Products.FileSystemSite.DirectoryView this import will be removed in Silva 2.3
  from Products.Silva.fssite import registerDirectory
Warning: Missing factory for Products.ProxyIndex.ProxyIndex ProxyIndex
Warning: Missing factory for Products.ProxyIndex.Expression Expression
New implicit rule detected Persistence PersistentMapping to Persistence.mapping PersistentMapping
/svn/silva-2.2/eggs/Products.SilvaNews-2.8b1-py2.4.egg/Products/SilvaNews/silvaxmlattribute.py:24: DeprecationWarning: translate is deprecated. Please import from silva.translations instead, this import location will be removed in Silva 2.3.
  from Products.Silva.i18n import translate as _
New implicit rule detected BTrees._IIBTree IIBucket to BTrees.IIBTree IIBucket
New implicit rule detected BTrees._OOBTree OOBucket to BTrees.OOBTree OOBucket
Committing changes.
Found new rules:
{'BTrees._IIBTree IIBTree': 'BTrees.IIBTree IIBTree',
 'BTrees._IIBTree IIBucket': 'BTrees.IIBTree IIBucket',
 'BTrees._IIBTree IISet': 'BTrees.IIBTree IISet',
 'BTrees._IIBTree IITreeSet': 'BTrees.IIBTree IITreeSet',
 'BTrees._IOBTree IOBTree': 'BTrees.IOBTree IOBTree',
 'BTrees._IOBTree IOBucket': 'BTrees.IOBTree IOBucket',
 'BTrees._OIBTree OIBTree': 'BTrees.OIBTree OIBTree',
 'BTrees._OIBTree OIBucket': 'BTrees.OIBTree OIBucket',
 'BTrees._OOBTree OOBTree': 'BTrees.OOBTree OOBTree',
 'BTrees._OOBTree OOBucket': 'BTrees.OOBTree OOBucket',
 'Persistence PersistentMapping': 'Persistence.mapping PersistentMapping'}

[/end console]

and here are two links to the visuals that I receive from the Silva site
http://al-xo.com/troubleshoot/Silvasite.jpg
http://al-xo.com/troubleshoot/silvasiteerror.jpg

Revision history for this message
mallow (princemallow) said :
#3

I didn't notice the traceback, perhaps that can help too

Traceback (innermost last):
  Module ZPublisher.Publish, line 119, in publish
  Module ZPublisher.mapply, line 88, in mapply
  Module ZPublisher.Publish, line 42, in call_object
  Module silva.core.views.views, line 108, in __call__
   - Class: Products.Silva.ExtensionService.ManageExtensions
   - Object path: /silva/service_extensions
   - Object type: Silva Extension Service
  Module grokcore.view.components, line 55, in __call__
  Module zope.publisher.publish, line 108, in mapply
   - __traceback_info__: <bound method ManageExtensions.update of <Products.Silva.ExtensionService.ManageExtensions object at 0xcd3790c>>
  Module zope.publisher.publish, line 114, in debug_call
  Module Products.Silva.ExtensionService, line 388, in update
  Module Products.Silva.ExtensionService, line 114, in upgrade_all
  Module Products.Silva.Root, line 173, in upgrade_silva
  Module silva.core.upgrade.upgrade, line 181, in upgrade
  Module silva.core.upgrade.upgrade, line 126, in upgradeObject
   - Object Info:
      {'object': <Silva Root instance silva>,
       'object_path': '/silva',
       'upgrader': <Products.SilvaNews.upgrade_28.SNNRefresher object at 0xad9680c>}
  Module Products.SilvaNews.upgrade_28, line 36, in upgrade
  Module Products.Silva.ExtensionService, line 69, in install
  Module Products.Silva.ExtensionRegistry, line 268, in install
  Module Products.SilvaNews.installer, line 19, in install
  Module silva.core.conf.installer, line 63, in install
  Module Products.SilvaMetadata.MetadataTool, line 222, in addTypesMapping
  Module Products.SilvaMetadata.MetadataTool, line 235, in addTypeMapping
  Module Products.SilvaMetadata.TypeMapping, line 65, in editMappings
  Module Products.SilvaMetadata.TypeMapping, line 29, in setDefaultChain
  Module Products.SilvaMetadata.TypeMapping, line 117, in verifyChain
  Module Products.SilvaMetadata.TypeMapping, line 111, in getMetadataSets
  Module zope.component._api, line 206, in getUtility
ComponentLookupError: (<InterfaceClass Products.SilvaMetadata.interfaces.IMetadataService>, '')

Revision history for this message
Sylvain Viollon (thefunny) said :
#4

Ok, so zodbupdate worked fine. The bug you have is because you have SilvaNews installed.

https://bugs.edge.launchpad.net/silva-news-network/+bug/584201

I will do a release of 2.2 in the coming weeks to fix this.

Revision history for this message
mallow (princemallow) said :
#5

Awesome, thank you so much for the help Sylvain I greatly appreciate the time you took out for me. Looking forward for the update :D

Can you help with this problem?

Provide an answer of your own, or ask mallow for more information if necessary.

To post a message you must log in.