glmark2

Is there a way to report failed benchmarks on Android?

Asked by Alvaro Moran

On some Android devices I have been eble to pop up the Glmark2 app and run the complete benchmark list, having the score at the end.
The issue is that some of the devices fail to render one of the tests (eg.: terrain), but instead of reporting it and continuing with the following benchmarks, it silently ends the run and goes back to the main screen.
Is there any way to avoid so? It would be great so Glmark could be used in an automated manner.

Question information

Language:
English Edit question
Status:
Solved
For:
glmark2 Edit question
Assignee:
No assignee Edit question
Solved by:
Jesse Barker
Solved:
Last query:
Last reply:
Revision history for this message
Best Jesse Barker (jesse-barker) said : 		#1

Terrain is a tricky one as it depends upon support for texture samplers in the vertex shader (which is not a required feature of OpenGL ES 2.0), so it will not run on all hardware. We are working on a couple of things that will help in this area. If nothing else, for 2012.08, we will be ensuring that if a test does not run for "legal" reasons (as with terrain) that it is not counted as part of the overall results. Of course, if a test fails to run, we will handle that differently :-).

Revision history for this message
Alvaro Moran (dirac3000) said : 		#2

Thanks Jesse Barker, that solved my question.

Revision history for this message
Alexandros Frantzis (afrantzis) said : 		#3

"but instead of reporting it and continuing with the following benchmarks, it silently ends the run and goes back to the main screen."

This is not normal behavior. When a test fails to run for some valid reason (extensions missing etc), we should just move to the next. What you are seeing indicates that the app may have crashed. Please open a bug report including the android logcat output, so we can try to figure out what's causing this (it may be the app or a broken driver).

By the way, regarding automated testing, you can invoke the benchmark activity with the parameters you want independently of the main GUI. See the README.android file in the source tree for more (http://bazaar.launchpad.net/~glmark2-dev/glmark2/trunk/view/head:/README.android)

Revision history for this message
Alvaro Moran (dirac3000) said : 		#4

Well, this is not the case. It doesn't look like the application has crashed, because I get back to the main menu just after the jellyfish.
Here's the logcat output of a complete run (loading the application and then just clicking on Run).

I/glmark2 ( 2859): glmark2 2012.07
I/glmark2 ( 2859): OpenGL Information
I/glmark2 ( 2859): GL_VENDOR: Imagination Technologies
I/glmark2 ( 2859): GL_RENDERER: PowerVR SGX 540
I/glmark2 ( 2859): GL_VERSION: OpenGL ES 2.0 build 1.7.17.4958
I/glmark2 ( 2859): [build] use-vbo=false: FPS: 57 FrameTime: 17.544 ms
I/glmark2 ( 2859): [build] use-vbo=true: FPS: 57 FrameTime: 17.544 ms
I/glmark2 ( 2859): [texture] texture-filter=nearest: FPS: 57 FrameTime: 17.544 ms
I/glmark2 ( 2859): [texture] texture-filter=linear: FPS: 57 FrameTime: 17.544 ms
I/glmark2 ( 2859): [texture] texture-filter=mipmap: FPS: 57 FrameTime: 17.544 ms
I/glmark2 ( 2859): [shading] shading=gouraud: FPS: 57 FrameTime: 17.544 ms
I/glmark2 ( 2859): [shading] shading=blinn-phong-inf: FPS: 57 FrameTime: 17.544 ms
I/glmark2 ( 2859): [shading] shading=phong: FPS: 57 FrameTime: 17.544 ms
I/glmark2 ( 2859): [bump] bump-render=high-poly: FPS: 57 FrameTime: 17.544 ms
I/glmark2 ( 2859): [bump] bump-render=normals: FPS: 57 FrameTime: 17.544 ms
I/glmark2 ( 2859): [bump] bump-render=height: FPS: 57 FrameTime: 17.544 ms
I/glmark2 ( 2859): [effect2d] kernel=0,1,0;1,-4,1;0,1,0;: FPS: 49 FrameTime: 20.408 ms
I/glmark2 ( 2859): [effect2d] kernel=1,1,1,1,1;1,1,1,1,1;1,1,1,1,1;: FPS: 20 FrameTime: 50.000 ms
I/glmark2 ( 2859): [pulsar] light=false:quads=5:texture=false: FPS: 57 FrameTime: 17.544 ms
I/glmark2 ( 2859): [desktop] blur-radius=5:effect=blur:passes=1:separable=true:windows=4: FPS: 24 FrameTime: 41.667 ms
I/glmark2 ( 2859): [desktop] effect=shadow:windows=4: FPS: 57 FrameTime: 17.544 ms
I/glmark2 ( 2859): [buffer] columns=200:interleave=false:update-dispersion=0.9:update-fraction=0.5:update-method=map: FPS: 57 FrameTime: 17.544 ms
I/glmark2 ( 2859): [buffer] columns=200:interleave=false:update-dispersion=0.9:update-fraction=0.5:update-method=subdata: FPS: 56 FrameTime: 17.857 ms
I/glmark2 ( 2859): [buffer] columns=200:interleave=true:update-dispersion=0.9:update-fraction=0.5:update-method=map: FPS: 57 FrameTime: 17.544 ms
I/glmark2 ( 2859): [ideas] speed=duration: FPS: 57 FrameTime: 17.544 ms
I/glmark2 ( 2859): [jellyfish] <default>: FPS: 31 FrameTime: 32.258 ms

Then I get back to the main menu.

Revision history for this message
Alexandros Frantzis (afrantzis) said : 		#5

The main GUI and the benchmarks run in different processes, so a crash in the benchmarks doesn't affect the GUI. What do you get after the jellyfish result in the log (not necessarily just from "glmark2")?

Revision history for this message
Alvaro Moran (dirac3000) said : 		#6

You are right, I had an error after the jellyfish test! Here it goes:

[...]

I/glmark2 ( 2518): [jellyfish] <default>: FPS: 31 FrameTime: 32.258 ms
E/ERS_UTILS( 1547): File open error
E/ERS_UTILS( 1547): File description NULL value
I/DEBUG ( 1547): *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
I/DEBUG ( 1547): Build fingerprint: 'lge/p2_open_eur/p2:2.3.7/GWK74/LGP940-V10h-JAN-13-2012.4797CAA2:user/release-keys'
I/DEBUG ( 1547): pid: 2518, tid: 2529 >>> org.linaro.glmark2:glmark2 <<<
I/DEBUG ( 1547): signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr b2331ca0
I/DEBUG ( 1547): r0 00306368 r1 00b89bc0 r2 0083a170 r3 b2331c6c
I/DEBUG ( 1547): r4 0000005c r5 0083a170 r6 00306368 r7 00b89c18
I/DEBUG ( 1547): r8 00b89bc0 r9 00000000 10 00000000 fp 00000001
I/DEBUG ( 1547): ip 81decccc sp 64264760 lr 0000000c pc 81d651e4 cpsr a0000010

[...]

I don't understand which one is the file it can't open though...
Do you think it's a build issue or a bug that I should report?

Revision history for this message
Alexandros Frantzis (afrantzis) said : 		#7

Feel free to open a bug report. If it turns out it's not a glmark2 issue we can just close it as invalid.

Please attach a considerable portion of the android log around the failure, so we can get a better idea of what is going on.