Keith,

I fully respect and appreciate that this site is for DisplayFusion support. So why am I writing here anyway with a non-DisplayFusion question? Simply because you guys know more about multi monitor user experiences than anyone, and frankly I really need the help. The situation I am struggling with has literally preoccupied my schedule for the last four months!

I never had a problem in smaller configurations (say, less than 8 monitors). But when I upgraded 6 months ago to 9 monitors, the problems began. It's a system performance oddity thing occurring at a very low level in the OS, because mouse cursor latency and typeahead latency are the biggest symptems. It's as if when I add more monitors, some kind of interrupt conflict bug or something gets triggered.

This weekend's project was the "big upgrade", which failed miserably. The aim was to rearrange existing monitors and add a few more. The new configuration (had it worked) would have been 7 2K monitors plus 6 more 4K monitors, yielding a 78.4 megapixel desktop on 13 panels. Yes, I really do need that much scre3en real estate for my work.

For a long time I was thinking there was a defective SATA device on the bus and it was hanging the system entirely (including mouse cursor response) for intermittent periods of several dozen milliseconds.

The problem was very palpable at 9 monitors, but the system is just plain unusable (due to being unable to navigate the UI due to extraordinary mouse cursor latency) with 13 panels. Furthermore, at this point I am pretty sure the GPUs have to be the problem because I think I have literally replaced everything else including the motherboard, CPU, and all attached SATA devices!

I *think* the right conclusion here is that my NVIDIA GTX90's just aren't up to the task of running 13 monitors. Actually, I think they have way more than enough "horsepower", because just ONE card will run four 4K monitors perfectly, without a hitch. So the cards are up to the demands of serving the monitors they serve. But add more cards and more monitors, and something goes screwy and it's driving me freaking insane.

At this point I think the next move is probably to buy four brand new GPUs of a different model that might work better together with 4 cards running between 13 and 16 monitors. (I could actually use more if we could get this working).

Four new GPUs is close to $2k by themselves, but I am in Mexico and usually by the time you pay all the extra shipping and import duties, the price is just about double what it would be in USA. And if they don't work and I have to return them, I'm totally screwed - you never get the import duty tax back when you return something, and internationally shipping the return back would cost me another several hundred. So the point is, just buying four GPUs *hoping* it might solve my problem is a very expensive proposition.

Again, I know that supporting non-DisplayFusion monitor issues isn't your job, Keith. I'd be more than happy to make a $100 cash donation to DisplayFusion if you guys would be willing to reciprocate by putting your heads together and helping me solve this problem. What we're talking about is basically high-end Wintel PC builder's tech support. It has nothing to do with DisplayFusion but everything to do with issues your engineers know more about than anyone, and hey, a hunred bucks is a hundred bucks...

To summarize the issue, I have no problem paying $2500 or even more to buy four new graphics cards that solve my problem for once and for all. But without expert help, I would be spending $2.5k with a 50/50 that it would change nothing and I would be stuck with four more very expensive paperweights that are not practical to return. Surely your team must know exactly what the real limits are and what exact GPUs are best at driving the most monitors.

Very interesting: At <6 monitors, when you right-click the desktopp and choose display settings, the WIndows 10 Settings flash screen (Big blue screen with the gear icon that indicates "settings" in white). It only lasts a fraction of a second. But get to 9 monitors on 3 cards, and try that. You get the blue flash screen for a good solid TWO MINUTES of real time, then FINALLY the actual dialog for configuring monitors up. Try it with 13 monitors and four cards? The blue flash screen stays up for a good three minutes, then the Settings applet appears to crash and goes away from TaskManager. As if something overwhelmed it and it couldn't cope!

So that last problem got even more screwy when I added the fourth card for the 13th monitor, because I didn't have another matching card in stock. I installed an AMD Radeon 7870-based card, and got all 13 monitors working (in the sense you can "Identify" them), I could not configure the relative positioning of the monitors (setup multiple monitors) function because it seems that Windows just crashes and burns and cannot run its own display setting applet with that many monitors. The NVIDIA Control Panel works, albeit INCREDIBLY slowly, but it only works on the NVIDIA cards! So I got 12 monitors where the mouse would flow across borders, but I don't even know how to get to the 13th monitor because Windows display settings just won't run at all.

Oh, important point: I am NOT a gamer, and generally don't care about frame rates the gamers need. My application is live-updating financial charts (stock market stuff). It's lots and lots and lots of pixels, but they don't change often. The only reason I chose such high-end gaming GPUs in the first place was that these cards offered the ability to each drive four 4K monitors (3 DP, 1 HDMI at reduced refresh rate, which isn't a big deal for me). I am NOT using SLI - there are currently 3 of these GTX970 GPUs in the system. I was going to order a 4th because the first 3 can only drive 12 of my 13 monitors, but now I'm thinking I need four more of a completely totally different model that won't cause this interrupt conflict (or whatever is making my system unusably slow) when multiple cards and monitors are used together. The symptoms here seem to be that running 2 cards with 4 monitors each works fine, but 3 cards with 2 monitors each, you move the mouse, walk across the office park and have lunch, come back, sit at your desk, and THEN watch the mouse cursor move! At 13 monitors it can be almost that bad. At 9 monitors (what I failed back down to now), there is frequent and annoying latency in both mouse cursor movement and typeahead. In fact I typed that entire last sentenence with no reaction on the screen, yawned, waited a sec, then it all suddenly popped up at once.

Again, each card cannot possibly be overloaded because it runs the same number of monitors p0er card nust fine so long as the other cards are not there running other monitors.

Obvious next thought - must be something deficient in the system build - maybe a little 250W power supply can't drive all those cards? I welcome feedback, but it's pretty hard for me to believe that could be the problem. Here is the full system configuration:

MSI XPOWER X99 mobo
i7-5820K 6-core 140W CPU
128 GB RAM
Boot disk: Samsung M.2 drive
Virtual machine working disk: RAID0 array of (4) 256gb enterprise grade SSDs
Blueray drive
(3) GV-N970IXOC-4GD NVIDIA GTX970-based cards, driving 3 monitors each via DP, some also driving a fourth via HDMI
(1) AMD Radeon 7870-based card to run the 13th monitor, but FYI I have already removed this card entirely from the system in case there was an NVIDIA-AMD conflict, and it made no difference whatsoever. I have the very latest drivers from the NVIDIA website.
Corsair AX860 top-of-line 860W power supply. The entire CPU box seldom draws more than 200W total.
All systems and monitors on an array of four different APC UPS devices. All systems run on conditioned power at all times. (necessary here in mexico)
(5) Dell U2713HM 2K monitors
(3) LG 31MU97-B 4K monitors
(2) Dell P2715Q 4K monitors
(1) Asus PB287Q 4K monitor

This is all on Win 10, but frankly one of the reasons I'm sure it's hardware or a driver issue is that the earlier incarnation of this system was a 64GB X79-based build on Win 7x64 Pro. The exact problems were on Win 7. We "replaced everything" to make this problem go away - new mobo with high end CPU and memory, so only the peripherals would be common and everthing else would be new. The exact same problems carried over to the new system which was running a new motherboard and CPU on a new X99 chipset platform.

So the GPUs are really the only common part that is still in the system as this problem has lingered from Win 7x64 to Win 10 and all the disk drives have been replaced. It seems pretty clear that the next move is ditch the GTX970's and buy four identical cards (each capable of at least four DP/HDMI outs) that are KNOWN to work well together in multi-monitor configs. That's where I think your engineers could really help most. I would be guessing what four GPUs to spend $2.5 - $3k on my the time I pay the tax and get them to Mexico, where it's effectively non-refundable due to all the international shipping hassles of a return. I would be guessing whereas your engineers would be speaking from experience.

Keith, can you help? I'm really desperate... I'm serious about the $100 donation - time is money and obviously this is not a DisplayFusion-specific issue. I'm reaching out to you guys because nobody anywhere knows as much about you about multi-monitor issues.

In closing summary: I have to believe there is a guy on your team who could look at this and say "I have no idea what's wrong, but if he bought four model XYX cards, I'm sure he'd have no trouble at all driving 13 or even 16 4K monitors with them. That's the one-line bit of advice I need to take the $3k risk of ordering four more cards and getting the wrong ones. More than happy to send that $100 donation as soon as I see it work!

Thanks,
Erik
Thanks in advance!

I'm not Kieth but hopefully I can shed some light on your issue and point you at a (possibly cheaper) fix.

I think the real bottle neck in your system is your CPU.

The i7-5820k "only" has 28 PCI Express Lanes. Your typical graphics card will use 8 (or 16 depending on your setup) so when you put in 4 of those, you've maxed out the number of lanes your CPU has available and your graphics performance tanks. So, your options are, get a CPU with more lanes (the i7-5930k for example has 40) or get a motherboard with space for another CPU and another i7-5820k (giving you 56 lanes).

Bare in mind any other devices you have on the PCI-e bus (sound cards etc) will also be using a lane (or more) so you may already be at your performance limit with the cards you have in now. Which seems logical with the issue you are having with 9 displays.

So upgrade your CPU before spending all your money on graphics cards would be my advice!

Thanks so very much for taking the time to reply, @mjeffreys! I really appreciate the help - was seriously consideering just buying 4 new graphics cards!

One more question: I see that Amazon is willing to ship a i7-5960X to me here in Mexico, but I can't source a 5930 here. Is it safe to assume (since it costs twice as much) that the 5960X would be an even better choice?

Also, the reason I went with the 5820X in the first place: That whole series only supports 64GB according to the spec sheet, but a guy on the newegg forums reported having to trouble running 128GB on the 5820k, so I went with that for sake of "known good configuration". Do you think it safe to assume that the 5960X should also support 128GB even if the spec sheet says 64?

I just realized as I was typing that even though the guy said it worked, the 5820 on 128GB is technically unsupported. I will remove half the memory this evening and boot it up and 64GB and make sure that doesn't change anything...

Thanks again!!!
Erik

One more question... Do you know of any way I could test to verify that the problem is what you suspect? Say, with some sort of CPU benchmark tool that monitors lane usage or something? After Mexican import duties and shipping, the 5960X will cost me about $1500...

The number of lanes supported by both the 5930k and the 5960X is the same (40) so you wont see any massive change there. The L2 Cache might help you depending on your work load type. But honestly I would suggest for your workload trying hard to source a 5930k so that you can possibly go for a pair of those if you decide you want even more pci-e lanes down the road.

That being said if you have the money I don't think you'll be disappointed with the 5960X

As for the memory configuration. Yes I'd definitely retest with 64gb.

If you are legitimately needing 128gb i'd suggest looking server cpus (xeons) and getting a pair of them. Again most will suffer from the 40 pci-e limit (I think theres a couple chips that have 48 lanes) but they will definitely not suffer from any ram limits. So perhaps a nice server board and a pair of xeons (again waaaay cheaper than your all new gfx list) would be the way to go.

As for benchmarking the actual performance pci-e. I don't specifically know of any tool to do that. I think the closes thing you would be able to see is with GPU-Z? and watching the performance graphs in that while you've got things going on but none of those are specific to the pci-e lanes.

Also, assuming you have 10 minutes this video is a great explanation of pci-e lanes and how they'll affect you

https://www.youtube.com/watch?v=rctaLgK5stA

Thanks so much for all the fantastic information! It's extremely helpful!

Best,
Erik

@mjeffries and @Keith,

THANKS SO MUCH for the incredibly helpful advice, guys. This problem is causing me so much grief that I'm going to just get an 5930K and two NVS-810's. I'd much rather spend the $2500 and just solve this for once and for all.

Don't worry guys - I totally appreciate the caveats and disclaimers. If it turns out this configuration doesn't work, there will be no hard feelings and I'll still very much appreciate your kind advice.

Erik

@ET69 fingers crossed for you. Would you be so kind as to let us know if you are successful with your new config when you get the new parts?

Of course, I'll be happy to report back. Just so you know what to expect, I was unable to source these parts for direct shipment to me here in Mexico. So they are being Fedex'd from Newegg to a buddy of mine in New Hampshire today. He will repackage and ship to me, but the customs process usually takes several days.

So what I'm saying is, bottom line it's like to be the 1st of May before I have these parts in my hot little hands and can rebuild the system. It also has to be a weekend project because I can't tolerate the downtime when the markets are open.

One more thought, somebody really needs to write a book (or at least a good white paper) on building large multi-monitor systems. I am a former software CEO and technologist, so in theory I should be way smarter about this stuff than 90% of the population. But I was totally, completely on the wrong track until @mjeffries explained the PCI lane thing.

Keith, what a perfect marketing opportunity for you guys. A white paper or video all about the trade-offs of DisplayPort vs. HDMI, the perils of running 4K monitors (esp. LG) on DP cables longer than 6', all the PCI lane and number of video cards stuff. A thorough explanation of what you really need (it's easy to think "I should buy a high-end gaming card just to make sure the thing has enough horsepower to run all those monitors". As we know, it doesn't work that way and the features needed to run lots of monitors are completely different from those needed for gaming frame rates. Even something as simple as why you DON'T want SLI/Crossfire if you are building a non-gaming system that just needs screen real estate... And of course the big one - why you absolutely cannot live without DisplayFusion on your new system. Seems like a cool way to do a good deed for the world and get some really good PR at the same time, no?

Erik

Hey Guys!

First of all, I want to emphasize that I really appreciate the thoughtful advice, regardless of the outcome.

But unfortunately, we're looking at same story, different hardware.

Up and running now, with i7-5930K and (2) NVS-810. As soon as I connect more than 8 or 9 monitors, it's exactly the same story as before. The system is literally unusable due to mouse latency alone. Something as simple as changing one display's orientation from landscape to portrait in NVIDIA Control Panel takes about 7 minutes after clicking APPLY before the dialog saying you have 20 seconds to confirm the change. Click yes to confirm and it's >5 minutes (yes, MINUTES) of spinning blue "modern day hourglass" cursor before the system is usable again.

The whole PCI lane story made perfect sense, so I have no doubt it was good advice. But now that explanation has been eliminated. Two cards can't possibly consume more than 32 PCIe lanes, and the CPU supports 40. My understanding is the top-of-line MSI X99 XPOWER AC mobo supports as many cards as you want running in x16 mode.

Any more ideas? One thing on my mind is that I don't think the spec sheet for this CPU supports 128GB, which is what is installed (and recognized by Windows). My next step on my own volition will be to try downgrading to 64GB and see if the symptoms change. Right now it appears to recognize all the memory just fine, but who knows if some weird glitch appears on the bus when you run >64GB which is on the CPU's spec sheet. (The "word on the street" I found around the net is that the spec sheet is out of date and these CPUs work fine with 128GB, but the masses who say that stuff aren't running 13 monitors either...)

All suggestions welcome on what else to try. This is literally driving me nuts.

Erik

p.s. For the record I have no regrets about the purchase even if it didn't work out. These NVS810s seem better suited to my needs and I can put the old cards to work in my standby backup system.

Oh man, I'm so sorry! I really did think that the pci-e lanes were the route cause of your problem. I wonder if its a mother board issue now?

I know its rubbish debugging steps. But have you tried to see if you have the same issues with the same cards on the same pci-e slots?

Might be worth a try.

Hang on guys, the story gets even better...

In NVIDIA Control Panel, on the Setup Multiple Displays page, the connected displays are shown on a per video card basis. Get this: It sees FOUR NVS-810's, even though only two are actually installed! The connected displays are correctly identified, but NCP sees them on four different cards when they are obviously only on two!

Erik

@mjeffries, please don't apologize! Like I said, I REALLY appreciate your help, and I have no regrets. My application (trading financial markets) is extremely fault-intolerant, so having plenty of spare computer parts lying around really is a good thing.

My biggest problem is I'm down here in Mexico where everything takes a week and a customs hassle to import. I have a partner in New Hampshire who could build a system for me, but the only way to test whether it works is to have 13 high-end monitors to test with, and obviously most people including my partner don't have that kind of hardware just lying around.

I'm totally open to spending more money on a server motherboard, but it's going to be a huge undertaking. Can't get those kind of parts shipped direct to Mexico, so only way to do it is have my partner receive the parts in NH and then forward for another $100 Fedex fee...

Any other thoughts?

Thanks again!!!

Just had an idea... I know the gaming crowd has a whole bunch of benchmarking tools they use to tune their systems for max performance. I don't know anything about these tools and have never used them.

Seems to me like a performance tuning tool would make a good diagnostic aid here - obviously SOMETHING goes wrong when all the monitors are connected, so I would think a performance tuning tool would identify the bottleneck pretty quickly.

My problem is that although I was a software guy in a former career, I know absolutely nothing about the benchmarking stuff. Any pointers on which performance tuning tools (preferably freeware or cheapware) that might be good diagnostic aids would be a huge help.

Thanks,
Erik

UPDATE: Downgrading to supported 64GB config made no difference, as expected...

*** Request for KEITH LAMMERS (or anyone else who can help) ***

Keith, while I tremendously appreciate the generosity you and @mjeffrys have shown in trying to help me figure this thing out, the situation has reached a point where what would actually be most helpful is your assistance in identifying an expert I can PAY to solve this problem for me.

I'm sure that with your help here and searching forums I could figure it out on my own, but I just don't have time to screw around with this any more and really need to figure out a solution that will work without any more screwing around.

Obviously my strong preference would be to find an expert who could help me find the most economic path to fixing the system I already have, including all the new parts I just dropped $2k on. But honestly, if I can't find that my next choice would be to find someone who can sell me an entire system (just the tower case, no monitors) that is GUARANTEED by the seller to actually work on the first try and drive all my monitors. Even if I had to drop another $5k on that, it would be painful but worth it to make the problem go away.

A google search found this: http://www.multi-monitors.com/Sixteen_Monitor_Computers_s/5137.htm

I'm planning to call them Monday to see if they will let me hire one of their techs to debug my current situation and fix it. Do you know anything about this company? Do you know any other vendors who can help me that you would feel comfortable recommending?

Thanks,
Erik

Keith,

Thanks so much for the prompt reply!

Yes, I've been thru three mice, just to be absolutely sure that wasn't it. But the symptoms are also pretty clear: When the system is "going slow", even something like clicking APPLY to change a setting in NVIDIA control panel can take more than five MINUTES of hourglass cursor to complete. So when I get to more than X number of monitors (X is around 6 or 7, approximately, but I haven't tested to determine the exact number where it starts to go sideways.

System lag begins at startup, even with no apps running and minimal system resource utilization according to Windows performance monitor. If I reboot the system with only 4 monitors turned on, the problem goes away instantly.

I didn't even know you could add tabs to task manager. Will try what you suggest and report back.

Spoke to Don at multi-monitor.com. Super nice guy and he was happy to help all he could over the phone. But he didn't offer much hope. Said he didn't remember encountering these symptoms on any of their multi-mon builds, and their tech would probably be starting from ground zero. He said he's run it past his techs and get back to me if they had any ideas, but he didn't sound very optimistic.

I just need to stop letting this thing consume my TIME.

Thanks again!

Erik

Spoke with Don at multi-monitor.com. Super nice guy, but unfortunately he wasn't super optimistic that his company could help. Said they don't do much with 4K in their shop and my problem didn't sound familiar. He said he'd run it by his techs but wasn't sure they'd be able to help.

Working on the Task Manager and NVIDIA Support suggestions now.

Erik

vmware.exe is the biggest user of objects, at 533 User and 1,470 GDI. Still a full order of magnitude below thresholds you said get dangerous.

tws.exe is the biggest user of screen real estate, with something in excess of 20 separate financial market charts displayed at the same time. But it's still only using 257 User, 493 GDI.

Also, the symptoms start as soon as the system boots with >8 monitors attached, so I really don't think any app is doing it.

Feels to me like what I really need is to be monitoring SYSTEM resources with some kind of benchmarking tool, and comparing what that report looks right after boot-up with 6 monitors, and then with all 13.

Thanks,
Erik

Thanks, Keith, that's an excellent idea.

My partner is working the NVIDIA tech support angle. I've got a spare 256GB SSD lying around, and will definitely follow your suggestion. But I can't do that kind of surgery on my system during the trading week, so it'll have to wait for the coming weekend.

I'll report back on how it goes.

Erik

In case you guys are interested, below is excerpted from my email reporting results of the testing NVIDIA Support directed that we try. Please note that this was an email to my partner, so please ignore the parts asking him to check with Newegg on return status, etc.

---

Wow… Lots to report. I have a feeling the NVS810’s are going to have to go back, so we need to get on deciding that question for once and for all ASAP, so I can get the process going if so. Much to follow for the support case.

Instructions provided by Shahbaz from NVIDIA Support:

The specific link provided at wagnardmobile.com did not work but redicted to their home page. I was able to find latest version of DDU utility by navigating their menus.

Installed and ran DDU. There were no checkboxes to check near the NVIDIA logo as per the instructions. I am guessing this is a version issue. Clicked Clean and Restart. Got a flickering monitor as the drivers were un-installed.

System restarted in low res on MSFT default driver. OMG, the mouse difference was amazing. I’d literally forgotten what it feels like when it works! Mouse latency was absolutely flawless, but of course with no drivers we’re only running one monitor.

Shut down and removed GTX970, leaving only 1 NVS810 (8 connected monitors) in system, as directed. Started up, again in single-monitor mode using MS default driver.

Performed “clean install” of 362.13 per instructions. About halfway through the install process, the mouse problem was back, bad as ever! This is the first time I’ve ever reproduced the problem with only one graphics card installed in the system, but it’s also the first time I ever ran a single NVS810 with all 8 monitors hooked up. I believe this is important for several reasons:

•For the first time we have clear evidence that the problem is NOT the PCIe lanes, or another hardware conflict. This is the only card plugged into the PCI bus!
•Seems to prove the theory that this is all about number of monitors (perhaps number of desktop pixels?) connected, not the number of graphics cards connected
•The problem reproduced the instant the driver activated all the monitors. This was BEFORE setting any custom resolutions, rotating monitors into Portrait, or using “Setup multiple displays” to configure which one is next to which other one. In other words, my unusual combination of Portrait and Landscape-oriented monitors appears to have now been ruled out as a cause.

Ran MSInfo32 as directed. .NFO file attached.

Ran NVIDIA control panel to obtain requested (attached) screen grab. It sees TWO NVS-810s when only one physical card is installed. I believe this is intentional, i.e. I think the NVS810 is actually TWO GPUs that each run 4 outputs, and they are internally linked with SLI. So I THINK the business about one one card showing up as two is the intended design, but we should verify that with NVIDIA support!

Looked at these “two” NVS810s in Device Manager. It sees one as “PCIe slot 4, bus 5”, and the other as PCI slot 4, bus 7”. This makes perfect sense to me – there are two PCIe “bus” devices, both sharing a single PCIe slot. That perfectly matches my hypothesis that the NVS810 is two GPUs on a single physical card. This gets important later (see below).

At this point, the testing prescribed by Shahbaz is complete, and requested files are attached. When I ran NVIDIA Control Panel, it was taking several MINUTES of real time in hourglass cursor to apply any change. I did NOT rotate or rearrange the monitors from their default configuration before taking the screen grab because it would have taken an hour.

Lots of new info trying to get the system to run again:

Put your seatbelt on – found way more interesting diagnostic clues while trying to get back to a running system for next week than I did following Shahbaz’ directions.

Re-installed GTX970 so I can get all 12 monitors working for the coming week. Booted system but Device Manager reported seeing the “two” NVS810s, and a generic “Microsoft Standard Adapter”, i.e. default driver. I knew the fix was to install the GTX970 driver, but I hesitated to do that (so far – I wind up doing that much later – see below). But I reasoned that the NVIDIA guy was trying to get the cleanest, non-conflicting configuration possible, and I didn’t want to screw that up. I knew that if I installed the 2nd NVS810, it would not correctly recognize my Dell P2715Q’s as 4K-capable, and would only run them at degraded res, which is a real problem for me because I rely on the high res for a bunch of charts. But I figured I could spend the rest of the day figuring that out on my own, and not screw up what the NVIDIA guy was trying to do by installing another driver for the 970.

Get this… I installed the 2nd NVS810, and connected all 13 monitors. The system booted up, but NCP only showed 6 monitors connected! It also continued to see only two NVS810 devices. Assuming my prior hypothesis was correct, we’d have expected to see four. Hang on, it gets better. Now Device Manager shows only two NVS810s (not 4), but this time they are on PCIe Slot 6, bus 6 and PCIe slot 4, bus 12!

After puzzling on this I realized what I THINK is happening. When both NVS-810s are connected, suddenly the system only sees one of the two logical devices per card. If it’s only seeing the first four ports on each card, it would make perfect sense that there are only 6 total monitors visible. If we are only seeing two 4-port logical cards out of 4 total, it would make perfect sense that one of them has 4 monitors connected and the other only has two. After all, we’re not using 3 of the total outputs, so it makes sense. However, to verify this hypothesis would require seeing which monitors the system sees, then tracing all the cables to see if the ports they are connected to on the NVS810’s match up to the above theory. I didn’t have time/energy for that multi-hour project.

So at this point I conclude that the NVS810s just aren’t both going to work in this configuration. This is a critical point to stress with NVIDIA support, because if the NVS810s are not going to work out, we really need to drop everything and get them back to you. Remember I have one running in the system, so this means a tear-down during the trading week.

Downloaded and installed latest GEFORCE 365.10 driver. Monitors went all screwy during install, and two reboot cycles were required before Windows recognized both the “two” NVS810s (back to both showing PCIe slot 4, busses 5 and 7), and the GTX970. In this configuration I was able to eventually get back to where I started, i.e. the problem is bad as ever, but my system is running on 12 monitors and I am ready for the market week ahead.

I note that regardless of whether or not the GTX970 is installed, NCP seems not to allow “Span Mode” (part of PhysX) to be used on any devices attached to the NVS810. If you attach the GTX970, suddenly NCP will allow span configuration, but only for monitors attached to the GTX970. This is a problem because part of my plan for running on two NVS810s relied on Span working for 3 of the monitors. Without that feature, I’m dead in the water until we can get a resolution to VMware not offering the right monitor combos, and I don’t have much faith in that happening soon.

MY OVERALL CONCLUSIONS:

I’m getting really concerned about these NVS810s for the following reasons:

•They don’t appear to support PhysX/Span mode
•They refuse to drive my Dell P2715Qs any higher than 2560x1440. The GTX970 instantly recognizes them as 3840x2160-capable
•In general, my impression is that the NVS810 is designed for video wall applications and isn’t a particularly feature rich card other than number of outputs.
•I think it may only support 30hz refresh rates.
•It does not appear to support 4096x2160 resolution, meaning my 3 primary LG monitors are running at slightly degraded res well.

I think you should make it clear in the NVIDIA forum that we have no emotional attachment to the NVS810s, and that we got them because someone recommended only two cards for PCIe lane overloading fears. There aren’t many 8-monitor cards on the market, but there are plenty that drive 6. I’m starting to wonder if we should be seeking their advice on (3) 6-monitor cards that would better suit my needs? We should make it clear we don’t mind buying more expensive cards. The whole point of the NVS810’s was that we thought we needed to run all the monitors on no more than two cards for PCIe-lane concerns. At this point I think we’ve ruled that out as being the problem.

I’m not even sure what to do if we have to return the NVS810s. It took an entire weekend to break everything down and re-wire it from a combination of HDMI and DP to all mini-DP. Just no way I could do that on a weeknight, and if we are going to eventually wind up with cards that use mini-DP outs, I really don’t want to take everything apart TWO more times! The cards I have here (if we sent the NVS810s back) are all DP, not mini-DP.

Going to try local stores to see if I can find mini-DPxDP adapters. If not, we might just have to eat the cost of at least one NVS810 and return the other one.

Please double-check Newegg.com, and make sure the NVS810’s are not a replacement-only warranty item! No sake knocking myself out to get them back to you if we can only exchange them for more NVS-810s…

Thanks,
Erik

Hi Keith!

The NVIDIA support crew supplied a long list of things to try. None of them worked, but they consumed a couple of weekends trying... So the fresh-install-windows option is definitely still in the queue, probably this coming weekend's project.

Have you guys built systems (or know customers who have) with 2 NVS810's installed? I notice that the two GPUs on each card are SLI'd together internally, but there are no SLI connectors to join the two NVS810 CARDS together. I wonder if this has anything to do with why when we install 2 NVS810s at the same time, both NCP and Windows Device Mgr only see one GPU (and 4 monitors) per card...

Thinking seriously about trying to find suitable AMD cards, and just buying them on speculation that MAYBE this is an NVIDIA problem. The guy at multi-monitor.com said they build all their big multi-monitor system on AMD cards, not NVIDIA. Keith, got any advice on AMD cards that drive 6+ 4K outputs, preferably with miniDP connectors, which is what I've re-cabled everything for?

Thanks,
Erik

Keith,

Quick update... Your suggestion to just start from scratch with a fresh install of Windows 10 was spot on. What was causing the old system to freak out when >6 monitors connected remains a mystery, but the new Windows install fixed it. Also the new build recognized both NVS810s, and allows all 13 monitors to function at desired resolution, albeit at 30Hz.

Unfortunately the new system also seems to have broken VMware, which won't allow multi-monitor VMs on the new sys but still works on the old sys. This forced me to fail back to the old sys for this week, but this weekend's project will be to try to fix VMware on the new sys...

Thanks again,
Erik

Keith,

BIG NEWS... After several months of working every weekend trying to debug this and upwards of $25k spent on parts, rush shipping to Mexico, and a consultant in the states... I've solved the problem and it appears that DisplayFusion was the culprit all along!

The most frustrating thing is that I could swear that one of the very first things I tried was to remove DF (although I did leave the license and config files) to see if that would solve it. I don't remember the details - that was months ago.

But after considerable testing I've determined that stopping the DF processes makes the problem go away. Specifically, there are four DF processes:

DisplayFusion
DisplayFusion Hook App
DisplayFusion Hook App (32 bit)
DisplayFusion Service

The first one appears to be the culprit - stopping this process eliminates the mouse lag and performance issues.

As I reminder, everything works fine until you get more than 8 or so monitors (mostly 4k) on the system. Then it suddenly becomes borderline unusable.

I will miss DF's wonderful features, but obviously I cannot invest any more time/energy trying to get it to work on my production system. I've used IObit Uninstaller to completely remove all DF files from my system, and so far it appears to be back to working normally.

I cloned the boot drive in case you want me to test anything or collect any diagnostic information that might be of help to you in debugging this. But please understand (1) I can only do this on weekends due to the system being in a mission-critical application, and (2) I haven't had a weekend off in four months because of this problem. So it'll be a couple weeks before I can do anything on the cloned system drive, which still has DF installed.

Please know I have no ill-will. Despite this being the worst system debug problem I've ever faced, you were right there to help even when it seemed clear the problem had nothing to do with DF. So I really appreciate your efforts and remain a fan of DF's feature set, even if it turned out to be the culprit in this very expensive adventure.

Thanks,
Erik

I think this may be more complicated than meets the eye.

The very first thing I did in the beginning was to remove DF (but keep config files and license installed), and that didn't seem to change anything, so I re-installed it. Hence my report that DF was not the problem.

Now that I have removed DF from the system, the worst *symptom*, chronic mouselag, is completely gone. But the system still "feels screwy". This system is hard-wired to a gigabit router with 200mbps Internet service. Yet I cannot watch a streaming video without it pausing every few seconds. Yet the video streams fine to my laptop, which only has a WiFi connection theoretically many times slower than the wired connection. And the desktop is 128GB memory on a i5930k and a super-fast 2000MB/sec M.2 boot drive!

So my latest hypothesis is:

1. Some unknown program is causing major system-wide performance degradation
2. If you run DisplayFusion, the problem is badly exaccerbated with mouselag symptoms

Unfortunately, I wiped out the brand new (fresh Win10 install on clean disk) sytstem I was building in parallel, because I couldn't get VMware to work on that system and allow my VM to run on multi monitors, which works on the performance-broken system. Otherwise it would make sense to install DF on that and see whether DF broke it. My guess would be non - I think DF is badly exaccerbating some other problem.

Keith,

As of your last note, I had to just walk away - this issue had caused so much grief I couldn't cope any more!

But I really miss having DF on my system. I see you've had a few releases since then. What do you think - is it likely that the problem is fixed? Is it worth putting some time into trying again? If so should I go with the latest version being offered or is there a special link I need to a beta version still?

Thanks,
Erik