Issues with Handling multipart/form-data in Ignition SCADA SDK Rest API development

Module Development
1

Hello Ignition Community,

I am currently developing a REST API using the Ignition SCADA SDK and have encountered significant challenges when working with multipart/form-data requests. While all other API endpoints in my project are functioning as expected, handling file uploads has proven to be a roadblock.

Here’s what I’ve observed:

  1. File Size Inconsistencies:
  • Uploading smaller files (e.g., less than 3 KB) is unreliable. It works sometimes but fails at other times with the error:
    You cannot pipe to this stream after the outbound request has started.
    image
    image1492×592 30.8 KB

The files could be less than 50 MB.

What I’ve Tried:

  1. Multipart Configuration: I configured the MultipartConfigElement in my code as follows:
MultipartConfigElement multipartConfig = new MultipartConfigElement(
        BASE_DIRECTORY + File.separator + "temp", // Temp directory
        500 * 1024 * 1024L, // Max file size: 500 MB(for test)
        500 * 1024 * 1024L, // Max request size: 500 MB(for test)
        1 * 1024 * 1024 // File size threshold: 1 MB(for test)
);
req.getRequest().setAttribute("org.eclipse.jetty.multipartConfig", multipartConfig);

and

MultipartConfigElement multipartConfig = new MultipartConfigElement(
                    BASE_DIRECTORY, // Temp dir
                    -1L, // No max file size limit for testing
                    -1L, // No max request size limit for testing
                    0 // No file size threshold
            );
            req.getRequest().setAttribute("org.eclipse.jetty.multipartConfig", multipartConfig);

Gateway Configuration: I updated the .config file to increase Jetty limits:

wrapper.java.additional.23 = -Dorg.eclipse.jetty.server.Request.maxFormContentSize=104857600
wrapper.java.additional.24=-Dorg.eclipse.jetty.server.Request.maxFormKeys=20000
wrapper.java.additional.25=-Dorg.eclipse.jetty.server.MultiPartFormInputStream.maxParts=10000
wrapper.java.additional.26=-Dorg.eclipse.jetty.server.HttpConfiguration.requestHeaderSize=32768
wrapper.java.additional.27=-Dorg.eclipse.jetty.server.HttpConfiguration.outputBufferSize=32768
wrapper.java.additional.28=-Dorg.eclipse.jetty.server.HttpConfiguration.idleTimeout=300000

Code flow:
From mountRoutes method:

routes.newRoute("/upload")
                .type("multipart/form-data")
                .handler(MinimalFileUploadTest::testUpload)
                .method(HttpMethod.POST)
                .mount();

public class MinimalFileUploadTest {

    private static final LoggerEx log = LoggerEx.newBuilder().build("streamchat.gateway.FileController");
    private static final String BASE_DIRECTORY = "C:\\uploads\\temp"; // Ensure this exists and is writable

    public static JsonObject testUpload(RequestContext req, HttpServletResponse res) {
        JsonObject responseJson = new JsonObject();
        try {
            MultipartConfigElement multipartConfig = new MultipartConfigElement(
                    BASE_DIRECTORY, // Temp dir
                    -1L, // No max file size limit for testing
                    -1L, // No max request size limit for testing
                    0 // No file size threshold
            );
            req.getRequest().setAttribute("org.eclipse.jetty.multipartConfig", multipartConfig);

            log.info("Before getParts()");
            Collection<Part> parts = req.getRequest().getParts();
            log.info("After getParts(), Parts Count: " + parts.size());

            for (Part part : parts) {
                log.info("Part Name: " + part.getName());
                log.info("Submitted Filename: " + part.getSubmittedFileName());
                log.info("Content Type: " + part.getContentType());
                log.info("Size: " + part.getSize());
            }

            responseJson.addProperty("success", true);
        } catch (Exception e) {
            log.error("Error: ", e);
            responseJson.addProperty("success", false);
            responseJson.addProperty("message", e.getMessage());
        }
        return responseJson;
    }
}

Key Challenges:

  1. Handling multipart/form-data in Ignition SDK appears more complex than expected. I am unsure if additional configuration is needed or if this is a limitation of the Jetty server.
  2. The inconsistent behavior with small files is puzzling, as I would expect smaller uploads to work reliably.
  3. req.getRequest().getParts() Stuck: The main issue seems to originate from this line in the code, where retrieving file parts is unreliable or fails altogether.

Questions:

  1. Has anyone successfully implemented file uploads with multipart/form-data using the Ignition SDK? If so, could you share guidance or examples?
  2. Are there known limitations with Jetty in the Ignition Gateway that could affect multipart handling?
  3. What additional configurations or best practices should I consider to ensure reliable uploads for both small and large files?

I’m eager to hear your thoughts and suggestions. Any help or insights would be greatly appreciated!

Thank you in advance! :blush:

2

Not sure how to solve it exactly.

@PGriffith, could you kindly lend us your time? What is being done wrongly here?

1 Like