nutrition lookup speed and memory

It sounds like there's a memory leak in the nutrition code. I haven't had
the time to actively develop Gourmet in a while, but I can tell you that
last I was using it, I was using nutritional data and not seeing anything
like that. I assume the problem goes away if you disable the nutrition
plugin?

It is possible that something about your particular database hit a
corner-case bug that triggered the memory leak. You might see if you can
reproduce the bug starting w/ an empty database (just start gourmet up
pointing it to a new directory with gourmet --gourmet-directory=/tmp/foo).

Tom

On Mon, Aug 26, 2013 at 5:51 PM, Stephen Haffly <
<email address hidden>> wrote:

> New question #234662 on Gourmet:
> https://answers.launchpad.net/gourmet/+question/234662
>
> I really like Gourmet as a program but there are a couple of issues I am
> having.
>
> 1. When launching, there is a definite hesitation as it is loading
> nutrition information. Even when running from an SSD, there is a hesitation
> when it reaches the point of importing nutrition entries at 7400 of 7413.
> On an HDD, this is even more pronounced.
>
> 2. When trying to view nutrition information, Ifind with a new recipe that
> most of the ingredients, even common ones, do not have nutrition
> information listed, instead reading that "Missing nutritional information
> for 16 of 17 ingredients." for example in one that I am looking at for
> reference. If I click on the Edit button, then the real slowdown starts.
>
> Using my netbook (AMD dual-core C50 processor, 4 Gb RAM, 120 Gb SDD) as an
> example:
>
> Before editing:
> Memory use with Gourmet open and viewing a recipe is using 699 MiB (19% of
> 3.6 GiB) with Swap being 0 bytes of 3.4 GiB. The only other thing running
> in the foreground is a system monitor. As soon as I hit the button for
> Edit, memory use starts to balloon and the system becomes very
> unresponsive. The nutrition search window does not even open for a couple
> of minutes. Memory use balloons to fill virtually all physical memory and
> then starts to fill up the swap space.
> It is currently at 3.4 to 3.5 GiB (95.4 to 96.8% of 3.6 GiB) and swap has
> grown to 1.4 GiB (41.8% of 3.4 GiB and climbing).
>
> Since I started typing this, the search window for nutrition information
> still has not opened, and it has been over 5 minutes.
>
> What is causing this tremendous amount of memory usage? Is there any way
> to tame Gourmet's appetite for memory? I had previously eliminated all
> pictures from the recipes in order to try to reduce the size, but it does
> not seem to make any difference when it comes to the nutrition information
> memory use.
>
> While this is going on, both CPU cores are running 57 to 99%. This makes
> the rest of the system extremely slow.
>
> The version on the Netbook is gourmet-0.15.9-8.fc19.noarch
> The version on the desktop is gourmet-0.16.0-1.fc19.noarch
>
> I had hoped that the upgrade to gourmet 0.16-1 would have fixed this
> problem, but it exhibits the same bad behavior.
>
> While I like Gourmet, these problems are making the nutrition part of it
> virtually unusable.
>
> --
> You received this question notification because you are a member of
> Gourmet Recipe Manager, which is an answer contact for Gourmet.
>

Thank you. Disabling the nutrition plugin indeed speeds things up tremendously. I'll have to check the other situation. Since it is late now, I will check and post the results later.

I decided to do the test you suggested above. I started up Gourmet with an empty database and made sure to activate the nutrition plugin. It started up quickly and the nutrition information also loaded very quickly.

I then imported a couple of recipes from text files, one plain text and another in MealMaster format. The text file required a little tweaking of the codes but the MealMaster recipe imported with no tweaking needed.

This time, when I went to look at nutrition information, the startup was fast and there was not the huge memory bloat.

Okay, I can see where it is likely a problem with my database. If so, then how can I fix my database so it will not have this problem? I would hate to lose the recipes I have or to have to type all of them.

Could you send us your recipes.db database so we can investigate? You could e.g. subscribe to our mailing list on https://launchpad.net/~gourmet and send an email with your database attached there, or send it privately to Tom and me.

Stephen,

There's also a decent chance the problem lies somewhere in your nutritional
DB which isn't usually part of the export (for better and worse).

So, as a temporary workaround, you could try exporting all your recipes,
starting Gourmet up from a temp directory, importing the recipes, and
seeing how things run. Most likely, it will work at this point. If you
don't mind having lost your nutritional associations (the links between
ingredients and nutritional information), you're all set for the time being.

As far as figuring out what's going on w/ memory, Bernhard's right that
sending along your db might be helpful. It's not that there's a problem
with your data so much as that something in your data has made the program
hit a corner case that we don't run into often that causes a memory leak.

Tom

On Tue, Aug 27, 2013 at 5:26 AM, Bernhard Reiter <
<email address hidden>> wrote:

> Question #234662 on Gourmet changed:
> https://answers.launchpad.net/gourmet/+question/234662
>
> Status: Open => Answered
>
> Bernhard Reiter proposed the following answer:
> Could you send us your recipes.db database so we can investigate? You
> could e.g. subscribe to our mailing list on
> https://launchpad.net/~gourmet and send an email with your database
> attached there, or send it privately to Tom and me.
>
> --
> You received this question notification because you are a member of
> Gourmet Recipe Manager, which is an answer contact for Gourmet.
>

The database file is too large to attach to an Email. I am sending a link to a SpiderOak share room where I have shared the contents of my .gourmet folder. The link is:

https://spideroak.com/browse/share/Shared_Files_from_Stephen/Gourmet_Files_For_Thomas_and_Bernard

The password to access the folder is grmdb

p.s. When it asks for username and password, just leave the username blank and enter the password only.

I just tried something. I installed Sqliteman. I opened the recipes.db with it and then selected the option System/Vacuum and used the option to Vacuum All. I am guessing that this is the equivalent of a Pack command. After doing so, it seems to have sped up the loading. However, trying to open recipes still takes time. The first one was faster, but the second one took about 2 minutes to open. Memory use also swelled up to 5.9 of 7.8 GiB on my desktop system (I did not check before starting, but after closing the recipe and Gourmet, memory use was 1.3 of 7.8 GiB. (with Gourmet 0.16.1).

Another change: I looked at the recipes again. The first one had duplicates in the ingredients (probably from when it was first imported from a MealMaster file). I changed that and that seems to have helped. Once SpiderOak finishes uploading the changed recipes.db, you will be able to get the latest version using the instructions listed above.

It does not cure the problem of memory use as it was still swelling from 1.1 GiB without Gourmet running to over 4 GiB with a recipe open and trying to edit nutrition information.

Also, something I recently noticed is that trying to enter custom nutrition information does not seem to work. I was trying to enter information for Stevia In The Raw Baker's Bag, which has <1 grams carbs per the nutrition label for a 1 ts serving size, but which has 18.4 grams of carbohydrates per 20 gram quantity of which .6 grams are sugars. Since this information is not in the USDA database, I would need to enter it manually.

What is the structure of the nutrition information in the /usr/share/gourmet/FOOD_DES.txt file? It looks like that is where the nutrition information resides. If I can add the information there, then I would not need to use the custom entry button.

Also, what is the relationship between ABBREV.txt and FOOD_DES.txt? Do entries have to be made in both for nutrition information to show up properly?

Stephen,

It's been a while since I looked at the nutrition code, but I can tell you
that the ABBREV and FOOD_DES files are just the files that come directly
from the USDA. They are not part of Gourmet's database -- there the text
files Gourmet starts with and imports when it generates its database, which
is kept in recipes.db and updated there when you make changes or add custom
info.

Tom

On Thu, Aug 29, 2013 at 3:01 PM, Stephen Haffly <
<email address hidden>> wrote:

> Question #234662 on Gourmet changed:
> https://answers.launchpad.net/gourmet/+question/234662
>
> Stephen Haffly gave more information on the question:
> Also, what is the relationship between ABBREV.txt and FOOD_DES.txt? Do
> entries have to be made in both for nutrition information to show up
> properly?
>
> --
> You received this question notification because you are a member of
> Gourmet Recipe Manager, which is an answer contact for Gourmet.
>