PDA

View Full Version : Changes/Update for v2.0



_
06-14-2013, 05:00 AM
Need suggestions for open source starting points:

1) OpenCL benchmark
2) OpenGL 3D benchmark
3) Storage/IO benchmark
4) Very heavy multitasking

Changes for v2.0

1) Upload function with website league table (split by chipset)
2) Readjust scoring to 1000's rather than 100's to generate more split between results. This isn't something we could have expected before getting lots of results in :)

xmanrigger
06-15-2013, 05:18 AM
I for one am more than happy with the present state of the benchmark. I really like it for testing CPU/memory clocks, and it does a very good job of it. I know I am Jonny Nobody, but if I had a say, I would exclude these:
1) OpenCL benchmark
2) OpenGL 3D benchmark
3) Storage/IO benchmark

Realbench as it is, is in my opinion one of the better stress tests out there to test cpu, mobo, and memory overclocks. There are already a crapload of GPU tests out there. Ditch i/o. Lots of those two. Bare bones stress test. As Realbench is. .....Jus sayin'.

-The upload function is very nice for a couple reasons I can see. One, save myself and others the pain of having to make/post a screenie. And two, it can validate a score. Have Realbench linked to user name somehow if possible.

- A basic system info summary (freqs, main voltages) on upload would be nice for validation as well.

- The scoring with 1000s is a very good idea also. Tightens things up. Increases competition.

- Very heavy multitasking. Why the hell not? LOL. I want a system buster.


Just my two cents. And just that.

Arne Saknussemm
06-16-2013, 05:36 PM
Not sure how open source works.....open GL and CL are open I guess but the stuff written for/on them?......are any of these useful?

http://www.geeks3d.com/softwares/

(http://www.geeks3d.com/softwares/)I would like something that tests the capability of the GPU with regards to open GL/CL as pertains to it's use in CS6 etc.....like a real world uses and programs....

Storage I/O not interested either....

HiVizMan
06-16-2013, 09:37 PM
Folks often try and do to much - thing software suites that try to be all things to all people. And they end up being not much use for anything.

For gamers there are two things you want to know - can my system play games at this frequency (both CPU and ram) and can my GPU play the games at this OC. Build a benchmark that tests those things and does so with some accuracy and you have a winner.

B0UJI
06-17-2013, 06:49 AM
Error checking like OCCT for GPU tests would be a good idea too.

Henkenator68NL
06-17-2013, 04:31 PM
How about increasing readability of the submitted scores.. Red directly on Black is difficult to read, even more if you happen to be collor blind.Also a white paper about what the benchmark exactly does test...And a directly updating score table would be nice Otherwise its fine!

Detleg
06-17-2013, 07:14 PM
I would like something that tests the capability of the GPU with regards to open GL/CL as pertains to it's use in CS6 etc.....like a real world uses and programs..

X2

especially when I get my second 780 next week ;)

_
06-18-2013, 08:58 AM
Thanks for feedback all :)

With regards to color - we'll go light grey on black next revision.

Nodens
06-19-2013, 06:57 PM
1) OpenCL benchmark


Currently there's no OpenCL benchmark out there at all. And OpenCL (GPU Computing) is something very useful to people that do image editing, video processing/encoding and 3d rendering considering it boosts the speed of those applications by A LOT.



2) OpenGL 3D benchmark


GPU performance should be part of the system wide score I believe. It affects games (we're Republic of Gamers after all heh) and people doing image/video/3d work (all the main workloads for high end hardware:)).



3) Storage/IO benchmark


Since no one seems interested in Storage I/O, perhaps we don't need it. The current multitasking test is VERY heavily dependent on file i/o since it does random reads in order to compress so it is a good indicator of file i/o performance. Also if we do implement such a benchmark it would have to be read-only anyhow or people afraid of SSD unnecessary writes would probably refust to run RealBench.
My last note on this is that I've been thinking about this test for a while now and I can not think of any "real world" scenario of testing file i/o so that it fits with RealBench's needs. Any reads or writes we make, or artificial iops load would be synthetic in nature. We'd just read and write specific data just for the purpose of the test.



- A basic system info summary (freqs, main voltages) on upload would be nice for validation as well.


While I like this idea, this is quite problematic to implement. Sure, I can get current frequency from the operating system API, but reading voltages is a WHOLE other story. It requires me to write a driver to access Super I/O chips. Considering how many different chips are there, and different implementations of sensors etc etc it would be a full time job just to maintain this. It would be like writing something like HWInfo/SpeedFan/AIDA from scratch and supporting it for all possible hardware configurations. This is not really feasable unless there's an open library+driver that already does that, which I could use to read stuff on a higher level and hardware support would be limited to what that library+driver supports. I will look for it but I doubt there's something that I can use available..

HiVizMan
06-19-2013, 07:10 PM
I just want to say that this bench is one of the best stability checks I have yet to find. If any of my systems pass this bench then they do not crash.

Respect to you Nodens.

Arne Saknussemm
06-19-2013, 07:46 PM
Yes I agree! RealBench is a class act and the proposed additions will only make it better.....nice idea ROG and excellent execution Nodens! I can really see this being my go to system test.

Nodens
06-19-2013, 07:53 PM
Very glad to hear that mate! Much better than CPU cooking for 8 hours:p

Detleg
06-19-2013, 08:03 PM
Yes Sire, what they said +1

xmanrigger
06-20-2013, 10:36 AM
I just want to say that this bench is one of the best stability checks I have yet to find. If any of my systems pass this bench then they do not crash.

Respect to you Nodens.

Exactly what he said ^^^^^^^^^^



Currently there's no OpenCL benchmark out there at all. And OpenCL (GPU Computing) is something very useful to people that do image editing, video processing/encoding and 3d rendering considering it boosts the speed of those applications by A LOT.

GPU performance should be part of the system wide score I believe. It affects games (we're Republic of Gamers after all heh) and people doing image/video/3d work (all the main workloads for high end hardware:))..

I agree, GPU is part of any system. Thing is, as soon as you include this, then there would be others wanting a 3D test. Then this and that. Soon it becomes bloated.

This bench in function as is, is perfect for subsystem overclock stability testing. Bare bones. All that is needed.



While I like this idea, this is quite problematic to implement. Sure, I can get current frequency from the operating system API, but reading voltages is a WHOLE other story. It requires me to write a driver to access Super I/O chips. Considering how many different chips are there, and different implementations of sensors etc etc it would be a full time job just to maintain this. It would be like writing something like HWInfo/SpeedFan/AIDA from scratch and supporting it for all possible hardware configurations. This is not really feasable unless there's an open library+driver that already does that, which I could use to read stuff on a higher level and hardware support would be limited to what that library+driver supports. I will look for it but I doubt there's something that I can use available..

I obviously have no coding knowledge. You are right, voltage sounds like a cluster to try and implement. Basic ifo would be nice though.

Thank you for your time, effort, and knowledge put into this. You da man.

Nodens
06-20-2013, 08:04 PM
I agree, GPU is part of any system. Thing is, as soon as you include this, then there would be others wanting a 3D test. Then this and that. Soon it becomes bloated.


Including this is essentially a 3D test. And surely we're not going to bloat it or make a test in the terms of 3DMark. A simple OpenGL test that represents Graphics performance. For sure we're not going to bloat it or add things randomly.

My proposal is that we add OpenGL test as part of the "System Score" because GPU is an integral component of the PC.

Then OpenCL and Heavy Multitasking can be added as separate solo tests (much like the Stress Test Mode is right now).
This way you have a representative system score that factors all major components and additional tests that may be useful to you but you're not forced to run in order to get a system score. Any future test we may decide to implement can be added as a separate test, like that.

With this design in mind you keep the main "System Score" test to the point and just provide additional tests for people who want/need them (of course we'll be adding only things that make sense..we won't add a Pi digit calculation test just cause someone suggested it:p it would be beyond the scope/concept of the application--All these decisions fall to ASUS of course).

So no bloating anywhere:) We're determined to keep the main System Score test to approximately the same duration. Have in mind, tests were tweaked several times during development for duration ;)

twisted1
06-23-2013, 06:20 PM
Currently there's no OpenCL benchmark out there at all. And OpenCL (GPU Computing) is something very useful to people that do image editing, video processing/encoding and 3d rendering considering it boosts the speed of those applications by A LOT.


I may have missed something here but, how about CLBenchmark?

http://clbenchmark.com/

Nodens
06-25-2013, 07:15 PM
Sorry I did not know CLBenchmark existed:) Or it didn't last time I checked. There is another one as well now that I did some further research but it doesn't really matter as we think it's important to have such a test.

twisted1
06-26-2013, 04:34 AM
It's pretty nice, you can run GPU's CPU, onboard GPU if it has OpenCL support(I think GPU- OpenCL support came with Ivy, don't think Sandy has that)

And you can run multiple gpu's :D

Chewitt
06-26-2013, 11:28 AM
I have to agree more GPU would be helpful as although i heavily OC my CPU and RAM i dont ever really touch the GPU and it would be great to get a good readout on how my GPU is comparing to others with similar kit.

Nodens
06-26-2013, 07:24 PM
It's pretty nice, you can run GPU's CPU, onboard GPU if it has OpenCL support(I think GPU- OpenCL support came with Ivy, don't think Sandy has that)


Sandy Bridge supports it:)

KPRage
07-01-2013, 12:39 AM
Nodens,

Hope I am not replying too late.. This thought came to me as I am planning to write an elaborate thread on some of the basics..

One thing I observed was, there is no one tool which will test all your major components; CPU, MOBO, RAM, PSU and, GPU, for stability..

Is it possible to include tools which do this? I know its a long stretch and, there will be complications involved however, it would be the best if it can be done..:)

I am not much aware of the technical challenges involved in this.. However, I do understand though, it might be pretty tough especially, when trying to integrate third party tools.. :)

Hope this helps in any way..

Cheers,
KP

_
07-01-2013, 02:22 AM
We are definitely looking into a more varied stability test, but it probably won't cover absolutely everything you've listed.

KPRage
07-01-2013, 04:44 AM
Yep.. That's good news nevertheless Marshall.. Telling it since we see a lot of people blaming the mobo or, CPU while the issue might be PSU or something else (just an example)..

It would really awesome if we can make a tool which can include most, if not all (still hoping someone can come up with it ;) ), the testing tools, which will then be more suitable to solve most of the issues.

As usual, you guys are doing an awesome job.. Loving every part of ROG.. Cheers for that mate.. :)

Cheers,
KP

twisted1
07-03-2013, 06:54 AM
I know they support OpenCL on the CPU, didn't think sandy supported OpenCL on the IGP also, it was launched as a new feature with ivy but maybe they got driver support for it now.

JimmyH
07-22-2013, 01:14 AM
23715Can I ask a stupid question. What is meant to appear on the screen for a stress test, just this??

_
07-22-2013, 07:20 AM
Yes just that. The test has no visual output only command line.

alpinestar
07-31-2013, 01:57 AM
how about sandboxing all the applications so you do not have to see em all over the screen and disabling mouse for test time, adding 3d bench
and my CPU is barely at 40% usage so maybe high prioritizing all the applications you use for benching RAM READ WRITE SPEEDS + RANDOMS ,
or simply doing mathematical equations with a python which is very good for using direct hardware and not that hard to implement. prime95 is old but alot of ppl still use it for stability testing you could add python application to mimic what prime does with few lines of code :)
Oh and most important part please make the size smaller than my OS :rolleyes:

_
07-31-2013, 08:52 AM
> 40% in first test due to single core/memory focus, 100% in second test. It divides by thread use.
> Package will only get bigger as we've got to incorporate whole apps.
> Python is unfortunately not a typical user scenario
> Having everything on the screen is the whole point: it's meant to simulate a typical user scenario of apps running. :)

Nodens
07-31-2013, 11:18 AM
In addition:
Running the same complex equation over and over again is the definition of a synthetic benchmark. The point of RB is not to measure how fast you can calculate prime numbers, pi digits, fibonacci sequence numbers or whatever. It's supposed to show you performance in a real world scenario. See how fast your system performs in something that any user might want to do.. manipulate a few images, encode their favorite DVD, compress a big directory tree while watching their favorite movie.

If we wanted to make a synthetic benchmark, I would not even consider Python..It would still be writen in C++ and assembly, where required, for performance reasons.

We do not force scheduling priority on any subprogram. They are running at their default priorities. This is the realistic scenario. People competing in benchmarking are free to tweak scheduling priority as they see fit (same with all other OS level tweaks). This also allows people to see the impact of such tweaks on their systems (if there is any) and learn when to follow such practices and when not.

IM2L844
08-09-2013, 07:30 PM
How difficult would it be to create a bootable ISO that would run independent of the OS? Sorry if this has already been addressed. I didn't see it anywhere or, if I did, I didn't know what I was looking at.

To be more specific, I was wondering if a Windows 7/8 x64 PE with RealBench as portable app was a feasible idea. Since I only have a cursory understanding of such things, I didn't know if there would be any benching advantages or disadvantages to having something like that on a USB or not.

What started me thinking about it was stumbling across this:

http://technet.microsoft.com/en-us/library/hh824972.aspx

Nodens
08-12-2013, 06:00 PM
First of all it won't run on WinPE environment because it depends on some things that are not available on a PE environment. But even if we could make it run that way we would not be able to distribute it due to licensing issues.

At some point I plan on compiling a version for linux (once I get into coding linux versions of a couple of platform specific functions). When that happens I could make a bootable livecd iso with Realbench that would be distributable freely with no issue. Still, for benching, the read/write speed of the usb flash drive used would bottleneck the scores. But you can always copy it to ram drive or an SSD drive on the system before running it.

IM2L844
08-12-2013, 10:58 PM
Ah..., I see. I didn't know. Thanks, Nodens. I'm always impressed by the amount of hard work you guys must put in and the general unselfish willingness to help others by everyone in this community. It's rare and refreshing and it doesn't go unnoticed. I've already learned a great deal here and it's been a real pleasure. Thanks again to everyone involved in making this community what it is. There should be an ROG Hall of Fame.

Nothing but respect,
Nick

HiVizMan
08-13-2013, 09:03 AM
As long as there is far greater differentiation on scores, I think that three digit scores makes it quite tight and has to indicate a rounding up or down someplace in the process. Add a decimal and three places or at least two. :)

_
08-14-2013, 02:17 AM
As long as there is far greater differentiation on scores, I think that three digit scores makes it quite tight and has to indicate a rounding up or down someplace in the process. Add a decimal and three places or at least two. :)

We'll go to 4 or 5 digits next time. I remember reading something about the early days of MadOnion/Futuremark where they discovered people just love scoring big numbers.

Depending on time/Nodens efforts, we could run public betas with scoring tweaks to see how different methods pan out.

midnite
11-25-2013, 06:14 PM
is it me or does version 2.0 seem to run alot smoother and faster then 1.1 especially on AMD platforms?

Nodens
11-25-2013, 06:47 PM
There are optimizations all across. Please take any v2 feedback to the v2 beta thread.

This thread was meant for feature suggestions for v2 which has been feature-locked for a while now. So I will lock this thread now, as it serves no purpose.