Showing results 1 to 5 of 5

Thread: [AddIn Service] Background worked - write variables

  1. #1
    Join Date
    31.05.2018
    Posts
    10

    Default [AddIn Service] Background worked - write variables

    Hey,


    What is the recommended way for maximizing write operation throughput within the AddIn Engine?

    Let's consider the following scenario:


    • Runtime Service does multiple writes based on the data received from the external system
    • The data throughput is pretty large. Let's assume that we want to do 100 writes on 100 different variables every 100ms
    • The variable state should be verified beforehand (i.e. StatusBits or the value)
    • The writes are done within the background worker (i.e. we have a background thread constantly polling for data and once we receive it we want to invoke the write operation - each writes operation shall be invoked in parallel - for instance, we could spawn a list of tasks writing the variables as needed or invoke Parallel.ForEach() and then get the results and send the acknowledgment back to the overseeing system)



    For the sake of simplicity let's only consider internal variables for now. We notice significant performance drops when trying to write faster than 100ms - when we test out intervals ~250ms+ everything looks good, but as soon as we decrease the time interval between each consecutive write, the runtime seems to be freezing (if we decide to check the variable status before writing) or slowing down (if we skip checks and treat the writes as a "fire-and-forget" operations, which is not perfect).

    EDIT: We also tried implementing a BufferBlock which we would use a sort of a dedicated channel for write/read operations on IVariable object (essentially - push the messages to the buffer block, consume them FIFO, and then write the changes in Zenon sequentially on the same thread so we won't get the "cross-thread" overhead which we get when trying to get the variable via IProject.VariableCollection[variableName]), but it doesn't seem to solve the issue (although the access time definitely improved).

    Interestingly - when we're testing multiple write cycles on that particular set of variables and actually measuring the response times we see that the write time sometimes jumps (it's pretty random, but happens every single time) in a way that access times to IVariable jumps from ~5-10ms to ~100-200ms. I am 100% confident that nothing else is trying to access any of the IProject properties (including the VariableCollection) at that time, so it is pretty unexpected.

    Now, I am aware that the AddIn engine essentially wraps Zenon's COM components and that actual work happens in the RT's main thread, but how can we make this as fast as possible? What would be the best course of action here? Ideally we're aiming at some solution for writing dozens of variables in 100ms intervals in parallel.
    Last edited by mwieczorkiewicz : 14th February 2022 at 17:13

  2. #2
    Join Date
    28.02.2013
    Posts
    226

    Default Re: [AddIn Service] Background worked - write variables

    Hello mwieczorkiewicz,

    have you considered using the GenericNet driver for this task?

    https://github.com/COPA-DATA/DriverExtensions

    With that the variable communication can be handled in a more "native" way...

    Regards,
    Felix

    Quote Originally Posted by mwieczorkiewicz View Post
    Hey,





    What is the recommended way for maximizing write operation throughput within the AddIn Engine?



    Let's consider the following scenario:


    • Runtime Service does multiple writes based on the data received from the external system

    • The data throughput is pretty large. Let's assume that we want to do 100 writes on 100 different variables every 100ms

    • The variable state should be verified beforehand (i.e. StatusBits or the value)

    • The writes are done within the background worker (i.e. we have a background thread constantly polling for data and once we receive it we want to invoke the write operation - each writes operation shall be invoked in parallel - for instance, we could spawn a list of tasks writing the variables as needed or invoke Parallel.ForEach() and then get the results and send the acknowledgment back to the overseeing system)





    For the sake of simplicity let's only consider internal variables for now. We notice significant performance drops when trying to write faster than 100ms - when we test out intervals ~250ms+ everything looks good, but as soon as we decrease the time interval between each consecutive write, the runtime seems to be freezing (if we decide to check the variable status before writing) or slowing down (if we skip checks and treat the writes as a "fire-and-forget" operations, which is not perfect).



    EDIT: We also tried implementing a BufferBlock which we would use a sort of a dedicated channel for write/read operations on IVariable object (essentially - push the messages to the buffer block, consume them FIFO, and then write the changes in Zenon sequentially on the same thread so we won't get the "cross-thread" overhead which we get when trying to get the variable via IProject.VariableCollection[variableName]), but it doesn't seem to solve the issue (although the access time definitely improved).




    Now, I am aware that the AddIn engine essentially wraps Zenon's COM components and that actual work happens in the RT's main thread, but how can we make this as fast as possible? What would be the best course of action here? Ideally we're aiming at some solution for writing dozens of variables in 100ms intervals in parallel.


  3. #3
    Join Date
    31.05.2018
    Posts
    10

    Default Re: [AddIn Service] Background worked - write variables

    Quote Originally Posted by felixst View Post
    Hello mwieczorkiewicz,

    have you considered using the GenericNet driver for this task?

    https://github.com/COPA-DATA/DriverExtensions

    With that the variable communication can be handled in a more "native" way...

    Regards,
    Felix




    We actually considered the DriverExtensions for this task, but unfortunately our project's spec points the AddInEngine as the preferred way of implementing it (and basically we aim at the transparent integration so it's all independent of the selected driver).

    Are we out of options here (besides DriverExtensions)? I was able to kind of shorten the write/read time by limiting calls to the AddInEngine to the bare minimum, but the delays are still there. 



  4. #4
    Join Date
    28.02.2013
    Posts
    226

    Default Re: [AddIn Service] Background worked - write variables

    So not considering any status bit checks or other API stuff you are talking about 1000 SetValue API calls per second.
    COM-API calls need to by synchronzid to zenon main thread and might need up to serveral milliseconds.
    So in my eyes your approach will not work reliable.

    What is your use case all about? Can you also describe the surrounding conditions?
    From what kind of third party system are you forwarding data to what PLCs over what protocols?
    Are those PLCs/protocolls capable of this ammount of writes?

    Please note that also GenericNet driver variables can be forwarded to other drivers using allocations...

    (One (last) option would still be to distribute the load to multiple Service Engines that only process a subset of the required data.)

    Quote Originally Posted by mwieczorkiewicz View Post

    We actually considered the DriverExtensions for this task, but unfortunately our project's spec points the AddInEngine as the preferred way of implementing it (and basically we aim at the transparent integration so it's all independent of the selected driver).

    Are we out of options here (besides DriverExtensions)? I was able to kind of shorten the write/read time by limiting calls to the AddInEngine to the bare minimum, but the delays are still there.



  5. #5
    Join Date
    31.05.2018
    Posts
    10

    Default Re: [AddIn Service] Background worked - write variables

    The overall performance depends on many factors (system power, project configuration, third party task, etc.) and therefore generall statements are hard to make.
    Even more if you are trying to find the maximum throughput capacity in your special setting.

    So I recommend to test the approach using GenericNet and allocations to see if it is faster than the AddIn-approach.
    Creating new test values for 100 variables each 100ms is done guickly in a driver extension.
    Also creating that 100 allocations using the API.

    With that you can test the performance GenricNet -[allocation]->Siemens Driver-[network]->PLC quickly.
    If this brings better results than your AddIn approach I would go on an port your communication Logic from the AddIn to the DriverExtension to fully test your path.

    I would be happy if you share your results here!

    Thanks and best regards,
    Felix

    Quote Originally Posted by felixst View Post
    So not considering any status bit checks or other API stuff you are talking about 1000 SetValue API calls per second.

    COM-API calls need to by synchronzid to zenon main thread and might need up to serveral milliseconds.

    So in my eyes your approach will not work reliable.



    What is your use case all about? Can you also describe the surrounding conditions?

    From what kind of third party system are you forwarding data to what PLCs over what protocols?

    Are those PLCs/protocolls capable of this ammount of writes?



    Please note that also GenericNet driver variables can be forwarded to other drivers using allocations...



    (One (last) option would still be to distribute the load to multiple Service Engines that only process a subset of the required data.)

    Well... the use case is pretty simple - we have an external application which produces updates via ZMQ, which we then consume and assign the values to individual variables, the PLC will be a Siemens one, and from the docs it seems that Zenon's driver is capable of 100ms update time. So, ideally we aim at writes that could handle that maximum update time in concurrent writes. Obviously - we're looking at maximum theorethical data output, so real-world scenario is slightly different.

    Can allocations handle that amount of data? Technically we could work with that (and programatically generate those allocations in the Engineering Studio) but I am not sure if it will work as expected.
    Last edited by felixst : 15th February 2022 at 14:48

Similar Threads

  1. Write variable from background worker
    By debaru in forum VSTA
    Replies: 2
    Last Post: 5th July 2021, 14:05
  2. Creating variables in Scada addin.
    By lsimas3 in forum Add-In Framework
    Replies: 4
    Last Post: 27th May 2020, 15:11
  3. How change the state of variable in AddIn Runtime service
    By tsvoboda in forum Add-In Framework
    Replies: 21
    Last Post: 26th September 2019, 08:08
  4. Read zenOn variables from Windows Service
    By matteo.bernardon in forum Engineering Studio & Service Engine (former zenon Editor & Runtime)
    Replies: 4
    Last Post: 2nd November 2018, 14:35
  5. Read and write variables from plc with VBA
    By cibertoni in forum VBA
    Replies: 1
    Last Post: 22nd September 2014, 16:36

Posting Rules

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •