Creating A Parallel Processing Environment in Excel

[SDruley]

I have a very large program in Excel (.xlsm) that has a COM portal (xlQ) designed to bring in trading data. I have found that the process of bringing in data will not function when my code is running. Since the interface uses functions to insert market data into the Excel workbook, I guess it stands to reason that these cpu dependent cells may not calculate while code is running. I have to provide for some quiet time through the use of an on-time macro to allow the data to flow. This causes a problematic increase in the cycle time of getting data and doing something with it. I need to find an asynchronous approach that allows the data to process efficiently while my code is running.
In my vba program, I have experimented with DoEvents and PowerPivots but lack the experience to implement these to achieve a parallel processing environment.
So, my goal is simple. I want to be able to generate BUY SELL signals during a 10 second cpu intensive cycle. When I am finished with the cycle, then I will begin it again (loop) and grab the data from the COM without any interruptions. So, the DO LOOP runs without interruption because the COM data is seamlessly available.
Due to the difficulties I have experienced I was forced to come out of the loop, based on an on-time macro, then re-enter the loop after 5 seconds of quiet time so the COM interface could process. These 5 seconds have rendered by software useless since the trades are being implemented 5 seconds too late.
In order to process the billions of bits involved in the 10 second interval, I am using a 12 processor tower with 25G of memory and very fast clock speed. Even with this I am getting system warnings about cpu usage being too high.

What advice do you have?

[event21]

Hi -

If you can post your code as how you get the data using xlQ ( I believe it is via VBA codes also) and what are you doing after you get this data, then we might give some insight.

Probably, take out the xlQ processing in the VBA and run it somewhere like VBS script then set a time interval to run, then you can run your other VBA staff referencing the data grabbed from the system that has been dump in a text file or something..

Regards,
event

[SDruley]

Since my software is classified, let me add some commentary around the main issue of the xlQ data not calculating.
The xlQ COM add-in is a standalone system written in C+. It’s interface with Excel is in the form of built-in functions that capture specific stats from the stock market like “=xlqintraprice(H4, L15)” where H4 is the type of stock index and L15 is an index referring to a specific record in the xlQ data registry. So these are formulas that reside in the worksheet and they simply will not calculate when code is running. The only way they will calculate is if I put in my code the xlQ formulas and reenter them into each cell but this takes too long… this would be the same as editing the formula without making any changes then pressing enter.
The problem would be solved if the formulas provided by xlQ would calculate like any other cells would when automatic calculation is enabled and when code is running or not running.