SA-27398

Summary

Poor Performance while adding attachment.

After a Pega upgrade went live from PRPC 6.1 SP2 to Pega 7.1.9, users have reported significant issues, while adding attachments to cases from either attachment gadget or from add attachment flow action in actions drop-down.

User has reported the system hangs and takes approximately 5 minutes or more to upload the documents, for file size of 4MB or more, even drag and drop feature hungs and browser crashes if multiple files are dragged, with some files of more than 4 MB size.

It is found that Obj-Save method (write blob) in AttachFile activity takes significant time and might be the root cause.

Error Messages

Not Applicable

Steps to Reproduce

Attach a document larger than around 3 MB.

Root Cause

A third-party product issue.
While the peformance issue is seen during a step "Obj-Save", this is not a database issue or commit issue at the database level.
Thanks to thread dump, it is found the time was spend deflating the compressed content at the database level:

"WebContainer : 5" Id=1702 in RUNNABLE (running in native)
BlockedCount : 14, BlockedTime : -1, WaitedCount : 46467, WaitedTime : -1
    at java.util.zip.Deflater.deflateBytes(Native Method)
    at java.util.zip.Deflater.deflate(Deflater.java:442)
      - locked java.util.zip.ZStreamRef@81a5ca04
    at java.util.zip.Deflater.deflate(Deflater.java:364)
    at java.util.zip.DeflaterOutputStream.deflate(DeflaterOutputStream.java:263)
    at java.util.zip.DeflaterOutputStream.write(DeflaterOutputStream.java:223)
    at java.io.FilterOutputStream.write(FilterOutputStream.java:109)
    at com.pega.pegarules.data.internal.access.CompressImpl.deflate(CompressImpl.java:144)
  The performance issue is seen at the JVM level. It appears the JVM does not perform properly with the GC policy "balanced" for deflating stream larger than 3 MB.

For reference, the environment used was:

• WebSphere Platform 8.5.5.7
• Host Operating System is Linux, version 2.6.32-642.el6.x86_64
• Java version = 1.7.0, Java Runtime Version = pxa6470sr9fp10-20150708_01 (SR9 FP10), Java Compiler = j9jit26, Java VM name = IBM JVM

Resolution

After switching the GC policy from balance (Xgcpolicy:balanced) to gencon (via JVM setting within the Websphere console), the performance issue disappeares.