Results 1 to 3 of 3

Very slow process, sorting large table to identify range to apply array formulae

Threaded View

  1. #1
    Registered User
    Join Date
    07-10-2014
    Location
    England
    MS-Off Ver
    Office 2010
    Posts
    13

    Question Very slow process, sorting large table to identify range to apply array formulae

    Hi

    I have written a process to help me accomplish the below, but is very slow to run on my current dataset.

    Import information about delivery of shipments from an XML file generated from an SQL database (This is the only option for importing the data but is not the main reason for the slowness of the process)
    Check current data to see if either the name of who signed for goods or the date for when the goods were signed for are on the table containing the imported information by adding a formula to two columns at the end of my data (one for name, one for date)
    First sort by a column that contains a formula to let me know if the shipment on that row is on the imported data, there isn't already a name for who signed for the goods, and one exists on the imported data.
    Then sort by this column A-Z (Values from the Formula are N or Y)
    Then for first Y row to lastrow, apply an array formula.
    The array formula is too long to be added in one line, so I have to add it as multiple parts using .replace. I found that sometimes the .replace didn't work, so I added in waits to let it catch up.
    Then repeat the same process for the date column

    It might be easier to show my code and a duplicate file that imitates the layout of what I'm trying to achieve. There is a lot more to the file that may be slowing this down, but I've only duplicated the parts relevant to this process.
    Current process (I do not comment my work very well):
    Sub Add_PODs()
        Application.ScreenUpdating = False
        Dim CS As Worksheet
        Set CS = ActiveSheet
        Sheets("Shipment Data").Activate
        'Import Data
        ActiveWorkbook.XmlMaps("WestcoastPODs").DataBinding.Refresh
        Dim lastrow, firstPOD, firstList As Long
        Sheets("PODs").Range("Westcoast_PODs[#All]").RemoveDuplicates Columns:=Array(1, 2 _
            , 3, 4), Header:=xlYes
        ActiveWorkbook.Worksheets("PODs").ListObjects("Westcoast_PODs").Sort.SortFields _
            .Clear
        ActiveWorkbook.Worksheets("PODs").ListObjects("Westcoast_PODs").Sort.SortFields _
            .Add Key:=Range("Westcoast_PODs[[#All],[Westcoast Reference]]"), SortOn:= _
            xlSortOnValues, Order:=xlAscending, DataOption:=xlSortNormal
        With ActiveWorkbook.Worksheets("PODs").ListObjects("Westcoast_PODs").Sort
            .Header = xlYes
            .MatchCase = False
            .Orientation = xlTopToBottom
            .SortMethod = xlPinYin
            .Apply
        End With
        firstPOD = Sheets("Calculations").Range("AC14").Value
        'Check which jobs are on the POD list and which jobs can be permanently ignored now
        With Sheets("Shipment Data")
            lastrow = .Cells(.Rows.Count, "A").End(xlUp).Row
            .Range("P" & firstPOD & ":P" & lastrow).FormulaR1C1 = "=IF([@Signed]<>"""",""N"",IF(ISNUMBER(INDEX(Westcoast_PODs[Westcoast Reference],MATCH([@[PL Job Number]],Westcoast_PODs[Westcoast Reference],0))),""Y"",""N""))"
            .Range("Q" & firstPOD & ":Q" & lastrow).FormulaR1C1 = "=IF(ISNUMBER(INDEX(Westcoast_PODs[Westcoast Reference],MATCH([@[PL Job Number]],Westcoast_PODs[Westcoast Reference],0))),IF(OR([@[Date Delivered]]>(INDEX(Westcoast_PODs[Min POD],MATCH([@[PL Job Number]],Westcoast_PODs[Westcoast Reference],0))),[@[Date Delivered]]=""""),""Y"",""N""),""N"")"
            .ListObjects("WestcoastData").Sort. _
            SortFields.Clear
            .ListObjects("WestcoastData").Sort. _
            SortFields.Add Key:=Range("WestcoastData[[#All],[Replace Name]]"), SortOn:= _
            xlSortOnValues, Order:=xlAscending, DataOption:=xlSortNormal
            With .ListObjects("WestcoastData"). _
                Sort
                .Header = xlYes
                .MatchCase = False
                .Orientation = xlTopToBottom
                .SortMethod = xlPinYin
                .Apply
            End With
        End With
        'Set Name/Date Formulas for jobs on POD list and set cells as values
        Dim DateFormulaPart1, DateFormulaPart2, DateFormulaPart3 As String
        Dim NameFormulaPart1, NameFormulaPart2, NameFormulaPart3, NameFormulaPart4 As String
        firstList = Sheets("Calculations").Range("AC15").Value
        NameFormulaPart1 = "=IF(LEFT([@[Consignee Name]],33)=""Clarity Computer Distribution Ltd"",""Y_Y_Y()"",IF(LEFT([@[Consignee Name]],12)=""Derry Morgan"",""Y_Y_Y()"",IF(LEFT([@[Consignee Name]],17)=""Primeline Express"",""Y_Y_Y()"",X_X_X())))"
        NameFormulaPart2 = "Internal Job"
        NameFormulaPart3 = "IFERROR(VLOOKUP([@[PL Job Number]],PODOverride,3,FALSE),Z_Z_Z())"
        NameFormulaPart4 = "IFERROR(INDEX(Westcoast_PODs[Signature],MATCH(1,(Westcoast_PODs[Westcoast Reference]=[@[PL Job Number]])*(Westcoast_PODs[Signature]<>""""),0)),"""")"
        With Sheets("Shipment Data").Range("J" & firstList)
            .FormulaArray = NameFormulaPart1
            Application.Wait (Now + TimeValue("0:00:03"))
            .Replace "X_X_X()", NameFormulaPart3
            Application.Wait (Now + TimeValue("0:00:03"))
            .Replace "Y_Y_Y()", NameFormulaPart2
            Application.Wait (Now + TimeValue("0:00:03"))
            .Replace "Z_Z_Z()", NameFormulaPart4
            Application.Wait (Now + TimeValue("0:00:03"))
        End With
        With Sheets("Shipment Data")
            If firstPOD > 3 Then
                .Range("Q2:Q" & firstPOD - 1).Value = "N"
            End If
            lastrow = .Cells(.Rows.Count, "A").End(xlUp).Row
            .Range("Q" & firstPOD & ":Q" & lastrow).FormulaR1C1 = "=IF(AND([@[Date Delivered]]>0,[@Signed]<>""""),""N"",IF(ISNUMBER(INDEX(Westcoast_PODs[Westcoast Reference],MATCH([@[PL Job Number]],Westcoast_PODs[Westcoast Reference],0))),""Y"",""N""))"
            .ListObjects("WestcoastData").Sort. _
            SortFields.Clear
            .ListObjects("WestcoastData").Sort. _
            SortFields.Add Key:=Range("WestcoastData[[#All],[Replace Date]]"), SortOn:= _
            xlSortOnValues, Order:=xlAscending, DataOption:=xlSortNormal
            With .ListObjects("WestcoastData"). _
                Sort
                .Header = xlYes
                .MatchCase = False
                .Orientation = xlTopToBottom
                .SortMethod = xlPinYin
                .Apply
            End With
        End With
        firstList = Sheets("Calculations").Range("AC16").Value
        Sheets("Shipment Data").Range("J" & firstList).AutoFill Destination:=Range("J" & firstList & ":J" & lastrow), Type:=xlFillDefault
        DateFormulaPart1 = "=IF(LEFT([@[Consignee Name]],33)=""Clarity Computer Distribution Ltd"",Y_Y_Y(),IF(LEFT([@[Consignee Name]],12)=""Derry Morgan"",Y_Y_Y(),IF(LEFT([@[Consignee Name]],17)=""Primeline Express"",Y_Y_Y(),X_X_X())))"
        DateFormulaPart2 = "WORKDAY([@[Ready Date]],1,PublicHols)"
        DateFormulaPart3 = "(IFERROR(VLOOKUP([@[PL Job Number]],PODOverride,2,FALSE),IFERROR(INDEX(Westcoast" & _
            "_PODs[Min POD],MATCH([@[PL Job Number]],(Westcoast_PODs[Westcoast Reference]),0)),"""")))"
        With Sheets("Shipment Data").Range("I" & firstList)
            .FormulaArray = DateFormulaPart1
            Application.Wait (Now + TimeValue("0:00:03"))
            .Replace "X_X_X()", DateFormulaPart3
            Application.Wait (Now + TimeValue("0:00:03"))
            .Replace "Y_Y_Y()", DateFormulaPart2
            Application.Wait (Now + TimeValue("0:00:03"))
        End With
        Sheets("Shipment Data").Range("I" & firstList).AutoFill Destination:=Range("I" & firstList & ":I" & lastrow), Type:=xlFillDefault
        With Sheets("Shipment Data").Range("I" & firstList & ":J" & lastrow)
            .Value = .Value
        End With
        ActiveWorkbook.Worksheets("Shipment Data").ListObjects("WestcoastData").Sort. _
            SortFields.Clear
        ActiveWorkbook.Worksheets("Shipment Data").ListObjects("WestcoastData").Sort. _
            SortFields.Add Key:=Range("WestcoastData[[#All],[PL Job Number]]"), SortOn _
            :=xlSortOnValues, Order:=xlAscending, DataOption:=xlSortNormal
        With ActiveWorkbook.Worksheets("Shipment Data").ListObjects("WestcoastData"). _
            Sort
            .Header = xlYes
            .MatchCase = False
            .Orientation = xlTopToBottom
            .SortMethod = xlPinYin
            .Apply
        End With
        CS.Activate
        Application.ScreenUpdating = True
        PODForm.Show
    End Sub
    One thing I tried was writing the table to an array, and running a loop through columns of the array to do the checks instead of using the helper columns. I used VBA to check each cell value against the imported data and if it needed to be replaced, I added the cell address corresponding to the array address to a range variable (using union after the first address).
    However this took longer to do just one column than my current method for both columns, so I abandoned this approach.

    I have taken out all the additional sheets that are part of the main spreadsheet, but tried to leave the Calculations page that I have in to try and keep the effect of the other work that is going on at the same time. I believe the problem may lie in the fact that I have used formulas in the data table that are being recalculated constantly but I'm not certain if this is the main issue or not.

    If anyone could offer some suggestions on how to increase the speed of this process or a better way to run this overall, it would be greatly appreciated.
    Attached Files Attached Files

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Similar Threads

  1. Large table of data, need to process for call details
    By cwinchell2883 in forum Excel General
    Replies: 1
    Last Post: 12-13-2013, 03:09 AM
  2. Slow macro due to formulae being copied in a range of cells
    By Themd in forum Excel Programming / VBA / Macros
    Replies: 3
    Last Post: 07-29-2009, 04:00 PM
  3. Summarizing data and replacing slow array formulae
    By SM2009 in forum Excel Formulas & Functions
    Replies: 0
    Last Post: 01-16-2009, 04:35 PM
  4. Replies: 4
    Last Post: 11-08-2005, 04:50 PM
  5. How to apply rounding across a range of cells with other formulae
    By Steve T in forum Excel Formulas & Functions
    Replies: 1
    Last Post: 10-20-2005, 03:05 PM

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts

Search Engine Friendly URLs by vBSEO 3.6.0 RC 1