I have two nVidia GTX295 cards installed in my machine.

BOINC/GPUGRID are only using two of the cores. I'm on Vista x64 and have four monitors attached and the desktop extended onto all four. I have SLI and PhysX disabled, I removed the bridge between the cards and still the same.

My machine is a 4.2GHz water cooled QX9770 processor, 4GB 1800MHz RAM, 1TB disk space.

Do instructions exist for getting 4 cores to work in 9800GX2's or GTX295's?

Thanks!

Jason

Some completed now it appears.

........ wow Rapture - just ordered - so I'm a few days behind you. Are you pleased with the configuration?

P.

Unable SLI - than it works

Or he booted up, disabled SLI, but did not restart BOINC ...

BOINC would see the post boot SLI mode and not know that he deselected SLI ...
____________

I also have a machine that operates on Vista 64 and on every restart, Windows never recognizes the 2 graphic cards I have in the machine.

I set the option to prevent BOINC from starting on boot so I can go fiddle with the NVIDIA & desktop settings. In the NVIDIA control panel I have to disable the PhysX capability, then go to the desktop display settings and extend the monitor into monitor 3.

I believe that if the PhysX capability isn't disabled, the settings won't hold (extend into monitor X). After I have the monitor extended, I then go back into NVIDIA control panel and enable PhysX then select multiple monitors.

NVIDIA then recognizes both monitors and I start BOINC which will indicate
GPU(2) and supply the work for both cards.

I believe the last driver update I downloaded from NVIDIA was 180.84. Don't know if this helps, but this is the only 64 bit OS machine I have with dual cards.

My 32 bit Windows OS has no problem detecting both cards on boot up.

SLI is disabled, PhysX is disabled. I tried without the SLI strap installed, rolling back to 180.87 drivers, removing the four GTX295's from the Device Manager and letting Windows Vista x64 reinstall them and then reinstalling the drivers... Still only two GPU's. Apparently the first GPU in each card is being recognized for CUDA applications.

Windows sees four devices, Quad-SLI works fine for gaming...

I can remove one of the cards and Windows sees two GPU's and instead of Quad-SLI in the nVidia Control Panel I get "Multi-GPU Mode".

Windows sees the GPU's but I can't get BOINC *or* Folding at Home to see the second two GPU's.

jaf

SLI is disabled, PhysX is disabled. I tried without the SLI strap installed, rolling back to 180.87 drivers, removing the four GTX295's from the Device Manager and letting Windows Vista x64 reinstall them and then reinstalling the drivers... Still only two GPU's. Apparently the first GPU in each card is being recognized for CUDA applications.

Windows sees four devices, Quad-SLI works fine for gaming...

I can remove one of the cards and Windows sees two GPU's and instead of Quad-SLI in the nVidia Control Panel I get "Multi-GPU Mode".

Windows sees the GPU's but I can't get BOINC *or* Folding at Home to see the second two GPU's.

jaf

ON reflection , since it affects both BOINC and Folding, it is, as I said, more likely to be an issue with the CUDA drivers ... are you sure you have the latest?
____________

Paul,

I appreciate the help, I actually have three GTX295 cards and if I can get two to work I'm going to try and get the third fired up.

What's really weird is that if I put just a single one of my GTX295 boards in my machine, neither BOINC nor F@H will use the second core. F@H will try but it always immediately returns with UNSTABLE_MACHINE.

I've tried every nVidia driver version that supports the GTX295 that is publically available:

180.87 (first known GTX295 driver)
181.20 (current WHQL)
181.22 (beta made available tonight (1/16/09))
185.20 (leaked beta available on guru3d.com)

All of these drivers perform the same for me on my EVGA/nVidia 790i Ultra motherboard.

I've installed Windows 7 BETA on a second partition with 181.20 WHQL drivers. Same behavior.

I have spent hours in my BIOS playing with various settings and slowing my system way down just to be sure. No joy.

I'm not a CUDA dev so I can't know for sure, but everything seems to point at CUDA. However.. I've read forum posts from others have been able to get multiple GPU's working with F@H; some with EVGA boards like mine and some with other vendors' cards. That's why I spent so much time in my BIOS.

It's not a power problem. This machine has two supplies, a 650W Antec for my case fans and water pumps and a 1,200W Thermaltake Toughpower for the motherboard & video cards. Prior to the GTX295's I had water cooled tri-sli GTX280's in this machine that ran fine under full F@H load. Now I can't get a single GTX295 to fold on its second GPU for even a split second.

jaf

To my suspicious mind the key is that the system seems to recognize up to TWO processors ...

Why not four I don't know ...

BUt, it does if you only have one card.

Why NOT try the third card just for grins and see if it sees it at all?

It might surprise us and see all 6 processors then ... can't see 4, but can see 6 may be another clue (if it happens) ...

If it does not appear to BOINC THEN, well, at least we have another clue.

I doubt if it is settings for the simple reason that the OS and the tools see the two CPUs on the GPU cards. That is why I smell trouble with the drivers ... but who the heck knows ... you could send the cards to me to try ... and I am sure I would get around to sending them back in a year or two ... :)
____________

exchange twixt me and Rom:

Rom,

If the guy has only one card installed it is seen as two processors ...

I he installs the second, only one processor on each card is seen ...

SLI is not installed ...

Windows reports 4 CPUs on the two GPUs ...

The thread we are discussing this on is: http://www.gpugrid.net/forum_thread.php?id=653

Should you want to join in there ...

On Jan 16, 2009, at 8:01 PM, Rom Walton wrote:

Paul, thanks for the bug report.

To clarify, the GTX295 only reports itself as a single CUDA device, and you and others expect it to report itself as two CUDA devices.

Something is tickling the back of my brain that says that could be expected behavior if the internal stream/multiprocessor count doubles between the single processor vs. multiprocessor cards. Since there is already an internal SLI connection between the two graphics processors on a single card.

I would be interested to see what the coprocessor xml structure reported by BOINC reports for Both the single processor version vs. the multiprocessor version of the same chip.

I'm not in front of my computer at the moment, i'm on my cell phone. So I can't look up the source to see the excact field name.

----- Rom

One more depressing thought ... bad card ... you said you have three?

Try all three one at a time ... make sure they all get seen as we expect.

Card A, then Card B, Then card C

Try pairs to see if any of the three pairs show up as expected,

AB
AC
BC

Lastly, try all three (as: ABC) to see how they show up ...

We got feedback from Rom and he is one of the good guys (Paul's opinion) so, maybe he can get us somewhere ...

{edited for clarity}

{add}
Are you trying to run Folding or BOINC with GPU Grid?

Now I am confused when I review earlier posts ...
____________

Could you post your <coprocs> xml block for each of the configurations you have listed? It can be found in your sched_request*.xml file that is sent to GPUGRID.

It is what information the CUDA system returns to BOINC during detection.

It should look something like this:

<coprocs>
<coproc_cuda>
<count>1</count>
<name>GeForce 9800 GTX/9800 GTX+</name>
<req_secs>0.000000</req_secs>
<req_instances>0</req_instances>
<totalGlobalMem>536870912</totalGlobalMem>
<sharedMemPerBlock>16384</sharedMemPerBlock>
<regsPerBlock>8192</regsPerBlock>
<warpSize>32</warpSize>
<memPitch>262144</memPitch>
<maxThreadsPerBlock>512</maxThreadsPerBlock>
<maxThreadsDim>512 512 64</maxThreadsDim>
<maxGridSize>65535 65535 1</maxGridSize>
<totalConstMem>65536</totalConstMem>
<major>1</major>
<minor>1</minor>
<clockRate>1688000</clockRate>
<textureAlignment>256</textureAlignment>
<deviceOverlap>0</deviceOverlap>
<multiProcessorCount>16</multiProcessorCount>
</coproc_cuda>
</coprocs>

Could you post your <coprocs> xml block for each of the configurations you have listed? It can be found in your sched_request*.xml file that is sent to GPUGRID.

It is what information the CUDA system returns to BOINC during detection.
....

Created a new partition and installed Windows XP Professional x32 and fully patched up to SP3.

Boom. BOINC/GPUGRID worked the first time with one card installed, I haven't stuck a second one in yet, but I'm betting it'll work.

This time when I booted BOINC the message was this:

1/17/2009 1:04:06 AM||CUDA devices found
1/17/2009 1:04:06 AM||Coprocessor: GeForce GTX 295 (2)

It's the first time it's ever said "GeForce GTX 295 (2)", always before it did not have the " (2)".

Embarrassingly enough, I cannot find the work files for GPUGRID. I found BOINC, but where are your files for GPUGRID?

Sticking a second card in now.

jaf

Yep it worked:

1/17/2009 1:38:17 AM||CUDA devices found
1/17/2009 1:38:17 AM||Coprocessor: GeForce GTX 295 (4)

putting the third card in now. ;)

jaf

Argh, I damaged the third card. In all the handling I lightly scratched the back of the card and knocked off a tiny little resistor or capacitor (it's so small I can't tell which it is).

I'll ship it in for an RMA, but the 6 GPU test will have to wait.

jaf

Rom, should swing by in the morning, but if he does not I will pop him a reminder e-mail that we may have isolated the issue and ask him to look here ... but probably won't have too ... he good guy ... :)

Sadly we had the one death in the family ...

But, you cannot make eggs without hatching a few omelets ... or something like that ...
____________

All four GPU's have been cranking along merrily overnight. Worst core temp is 81'C, which is hot for sure but I can tell from the sound that the fans are not at 100% yet.

I now have a very good test bed to help isolate and debug this problem and more when my third card is swapped out. If you guys would like me to help in some way just let me know. I have driver 181.20 working swimmingly on four GPU's in Windows XP Pro x32, and on the same machine a different partition that fails in Windows Vista x64.

Interestingly I did a clean install of Windows 7 Beta x64 the other night and installed the 64-bit 181.20 WHQL driver and Windows 7 behaved exactly the same as my Vista intsall. This leads me to believe that a clean install of Vista x64 would not resolve the problem.

My current hypothesis is an x64-specific problem in CUDA with detecting second GPU's in multi-GPU slot cards.

jaf

My current hypothesis is an x64-specific problem in CUDA with detecting second GPU's in multi-GPU slot cards.

A couple of days ago I sent in a support request to EVGA. At the time I was thinking that perhaps the BIOS in the EVGA cards was at fault. Here's my original message:

I purchased three EVGA GTX295 Plus cards for the purposes of gaming and running Folding@Home's GPU client. While the cards work perfectly well for gaming, I'm finding it impossible to get the F@H client to run on the second GPU of the cards despite a tremendous effort debugging the problem (found here: viewtopic.php?f=52&t=7874). Others seem to be having similar difficulties with the EVGA boards, but there's at least one guy with XFX boards that is having no problems. The most noticable difference is the slight overclock of the EVGA boards and the fact that the XFX BIOS numbers end in .72 and .73 while my EVGA cards end in .90 and .92. Some detail can be found here: viewtopic.php?f=52&t=7936.

I realize that this is a very esoteric issue, but is there a possibility that I can be notified somehow when EVGA updates the GTX295 Plus's BIOS? Thanks for any help you guys can offer and thanks for really great products and service.


Here is their response:

I’ve received an update from my PM team regarding folding at home:

- NVIDIA is aware of the issue and is working on it.
- XFX will suffer this problem as well, it is a driver bug.

Once nvidia notifies us of an update will make sure our folding team is informed immediately. Unfortunately we do not have a ETA either on such drivers.