Path of Exile - Performance and Stability

Posted: 19 Sep 2019 10:37 PM PDT

Performance and Stability

Game performance and stability have been hot topics in the community recently, so we wanted to give an update about our understanding of the current status of several of the important issues, and to reassure you that we are working on them. For many of the issues, we haven't yet identified their causes, and we'll update with more information as soon as we have. We felt it was important to post this before the weekend so that you know that the concern has been heard.

Blighted Map Instance Server Performance
When we talk about instance server performance, we're referring to the performance of the game server software that is running on one of our server machines. Normally we can fit dozens of instances per CPU core, and they take a fraction of a millisecond to process your actions for the frame. Under extreme gameplay circumstances though, the instance server can slow down due to the calculations taking a full CPU core more than 33ms (the serverside frame time).

In Blight, there are a lot of monsters and they have a lot of life. What normally happens when you engage a pack of monsters is that your attacks quickly cull out the weak ones, leaving just a few strong targets. In Blight, however, the weak ones last for much longer. As you know, Path of Exile has a very complicated stat system with tens of thousands of different values that your character can have. Many of these are used in combat calculations and dozens of small buffs, debuffs and ailments are applied to monsters as you attack them. The more monsters there are, and the longer they survive your attacks for, the longer these calculations take. Blight combat is an order of magnitude more strenuous on our servers than regular map content.

Once we knew that we had serverside performance issues with Blighted Maps, we wanted to check whether we could improve this issue by adding more servers. We observed that on physical game servers with different numbers of game instances, the number of dropped frames per instance was not higher on more heavily-loaded servers. This showed that the problem is related to specific instances utilising an entire CPU core for more than 33ms, rather than the server itself having too many instances on it. Throwing more money/servers at the problem would not solve it.

We are working on this problem and trying to find a solution that will reduce the calculation complexity without compromising gameplay.

Minion Performance
We believe that the popularity of minion builds with 3.8.0 have caused a lot of the performance problems that players are encountering.

Minions are monsters, so all of the problems mentioned above apply to them. They tend to last a long time and there are a lot of them on the screen, all of which are getting all the various buffs and graphical effects that would normally apply to players.

Alongside the release of legion, we made many changes to how melee combat works in Path of Exile. Our changes intended to tighten up attack distances and to get stuff to feel right had the side effect of slowing down pathfinding and causing a few other small performance regressions. We didn't notice at the time how much extra cost these added to minions.

With the 3.8.0 release making minion builds very popular, all of these performance problems have become a large issue for anyone playing minions, or in a party with minions.

These issues are a high priority for us.

Graphics Driver Crashes
While playing certain content (e.g., Betrayal missions), some users experience a graphics driver crash where the screen goes black for a while, and then comes back with a Path of Exile "Map" error. Recent versions of the Nvidia driver including the most recent, 436.30, have this issue. We can't reproduce the problem on Nvidia driver version 419.17. We don't yet know what Path of Exile is doing to cause the crash, but the graphics driver should never crash regardless of what the game does. We have contacted Nvidia to bring it to their attention.

(A few AMD users have reported similar problems but we haven't been able to reproduce this and it doesn't appear to be on the same scale).

Instance Crashes
Instance crashes are when the game instance server crashes while you're playing, disconnecting you back to the login screen. Since Blight's release, we have deployed many hotfixes that address various instance crashes and the number of crashes is dropping by the day. Today we have deployed a hotfix that resolves more than half of the remaining instance crashes. We are still working on the remaining ones, focusing attention on the most-frequent first.

Client Memory Usage
We have noticed unusually high memory usage, likely due to loading too many game assets. We believe this may be causing performance issues and we are investigating.

Engine vs Gameplay Performance
Recently there have been a lot of community posts criticising Path of Exile's engine for being slow or not optimised enough. We have spent several years improving the engine and have deployed these updates as soon as they have become ready. Path of Exile's engine is currently modern, lean and fast. But yet, in many situations, the game does not perform well at all.

Almost all of the performance problems that Path of Exile suffers from are related to gameplay systems and decisions we have made there. For the reasons described above, Blight is a league with performance problems. We made a conscious decision to try this type of league, and it's entirely appropriate to blame us and this decision for the poor performance that you may encounter while playing this content. We weren't expecting it to be this bad, and are working on fixing it.

No matter what engine we are using, Path of Exile would be slow in its current state due to the gameplay decisions we have made: calculating too many things on the server and rendering too many things on the client.

It may seem arbitrary and defensive for us to point out the difference between the engine and the gameplay systems, but the reason is that the engine is going to be part of Path of Exile forever, but the gameplay code comes and goes as things are added and removed. We don't want leagues to have performance issues, and this informs the decisions we make about future content development.

We're sorry that this post doesn't contain more definitive steps about exactly how and when we will solve these issues, but we wanted to communicate the status of these issues before the weekend. We are treating them as high priorities and are hoping to find and deliver solutions to many of them next week.