+ Reply to Thread
Results 1 to 7 of 7

Which is the quickest way to loop through large data sets

Hybrid View

newbi004 Which is the quickest way to... 05-03-2013, 09:10 AM
newbi004 Re: Which is the quickest way... 05-03-2013, 09:43 AM
tehneXus Re: Which is the quickest way... 05-03-2013, 09:47 AM
tehneXus Re: Which is the quickest way... 05-03-2013, 09:53 AM
newbi004 Re: Which is the quickest way... 05-03-2013, 10:06 AM
tehneXus Re: Which is the quickest way... 05-03-2013, 10:31 AM
newbi004 Re: Which is the quickest way... 05-03-2013, 10:38 AM
  1. #1
    Forum Contributor
    Join Date
    07-23-2012
    Location
    London, England
    MS-Off Ver
    Excel 2007
    Posts
    211

    Which is the quickest way to loop through large data sets

    Hi

    I have the following code:

    Sub GetSAPInfo()
        
        Dim lRow1 As Long, lRow2 As Long
        Dim ws As Worksheet
        
        Date1 = DateValue("01/02/2012"): Date2 = DateValue("01/05/2012")
    
        Set ws = Worksheets("Fuel 6050")
        ws.Select
        
        lRow1 = ws.Cells.Find(What:="*", SearchOrder:=xlRows, _
        SearchDirection:=xlPrevious, LookIn:=xlValues).Row + 1
        MsgBox "lRow1 = " & lRow1
        Set ws = Worksheets("Vehicles")
        ws.Select
        
        lRow2 = ws.Cells.Find(What:="*", SearchOrder:=xlRows, _
        SearchDirection:=xlPrevious, LookIn:=xlValues).Row + 1
        MsgBox "lRow2 = " & lRow2
        For i = 2 To lRow2
        Registration = Sheets("Vehicles").Cells(i, 1)
        
        For t = 1 To lRow1
        If Sheets("Fuel 6050").Cells(t, 11).Value = Registration And Sheets("Fuel 6050").Cells(t, 7).Value > _
        Date1 And Sheets("Fuel 6050").Cells(t, 7).Value < Date2 Then
        Fuel = Fuel + Sheets("Fuel 6050").Cells(t, 5).Value
        End If
        Next t
        'MsgBox i
        Sheets("Vehicles").Cells(i, 2).Value = Fuel
        Fuel = 0
        Next i
        
    End Sub
    I want to search through two large tables (am avoiding Pivot Tables for this exercise) to add cost values for selected data. First criteria is that I want to select data between two dates (Date1 and Date2 variables in code). Second criteria is that I have a list of 306 vehicle registrations in sheet ("Vehicles"). For each vehicle registration, I want to search a column of data in another sheet ("Fuel 6050") with 10690 rows to find the first selected vehicle registration. When the first one is found in "Fuel 6050", it selects the cost of the fuel spent during the set period.

    The above code works, but is interminably slow!

    Surely there must be a quicker way to do this?

    Cheers for any advice.

  2. #2
    Forum Contributor
    Join Date
    07-23-2012
    Location
    London, England
    MS-Off Ver
    Excel 2007
    Posts
    211

    Re: Which is the quickest way to loop through large data sets

    Would arrays be the answer here? If so, can anyone help start me off?


  3. #3
    Valued Forum Contributor tehneXus's Avatar
    Join Date
    04-12-2013
    Location
    Hamburg, Germany
    MS-Off Ver
    Work: MS-Office 2010 32bit @ Win8 32bit / Home: MS-Office 2016 32bit @ Win10 64bit
    Posts
    944

    Re: Which is the quickest way to loop through large data sets

    Hi,

    a general thing: Minimize traffic between VBA and the worksheet
    That means: Read the entire sheet with one line into an array and loop there instead of looping through cells, if you want to write data create an array and fill it, then write the entire array to the sheet with 1 line of code

    I started to write code, yould you provide a sample workbook?
    Please use [CODE]-TAGS
    When your problem is solved mark the thread SOLVED
    If an answer has helped you please click to give reputation
    Read the FORUM RULES

  4. #4
    Valued Forum Contributor tehneXus's Avatar
    Join Date
    04-12-2013
    Location
    Hamburg, Germany
    MS-Off Ver
    Work: MS-Office 2010 32bit @ Win8 32bit / Home: MS-Office 2016 32bit @ Win10 64bit
    Posts
    944

    Re: Which is the quickest way to loop through large data sets

    maybe this works already, if I assigned the sheets correctly:
    Option Explicit
    
    Sub GetSAPInfo()
        Dim aData1, aData2, aDataOut, Fuel, Registration
        Dim lRow1 As Long, lRow2 As Long, i As Long, j As Long
        Dim ws1 As Worksheet, ws2 As Worksheet
        Dim Date1 As Date, Date2 As Date
        
        Date1 = DateValue("01/02/2012")
        Date2 = DateValue("01/05/2012")
        
        Set ws1 = Worksheets("Fuel 6050")
        Set ws2 = Worksheets("Vehicles")
    
        With ws1
            lRow1 = ws1.Cells.Find(What:="*", SearchOrder:=xlRows, SearchDirection:=xlPrevious, LookIn:=xlValues).Row + 1
            aData1 = .Range(.Cells(2, 1), .Cells(lRow1, .UsedRange.Columns.Count)).Value2
        End With
    
        With ws2
            lRow2 = .Cells.Find(What:="*", SearchOrder:=xlRows, SearchDirection:=xlPrevious, LookIn:=xlValues).Row + 1
            aData2 = .Range(.Cells(2, 1), .Cells(lRow2, .UsedRange.Columns.Count)).Value2
        End With
        
        ReDim aDataOut(LBound(aData2, 1) To UBound(aData2, 1))
        
        For i = LBound(aData2, 1) To UBound(aData2, 1)
            Fuel = 0
            For j = LBound(aData1, 1) To UBound(aData1, 1)
                If aData1(j, 11) = aData1(i, 1) And _
                   aData1(j, 7) > Date1 And _
                   aData1(j, 7) < Date2 Then
                   
                    Fuel = Fuel + aData1(j, 5)
                    
                End If
            Next j
            aDataOut(i) = Fuel
        Next i
        
        With ws2
            .Range(.Cells(2, 2), .Cells(UBound(aDataOut) + 2, 2)).Value2 = Application.WorksheetFunction.Transpose(aDataOut)
        End With
        
    End Sub

  5. #5
    Forum Contributor
    Join Date
    07-23-2012
    Location
    London, England
    MS-Off Ver
    Excel 2007
    Posts
    211

    Re: Which is the quickest way to loop through large data sets

    Hi there

    Thanks for responding. I attach a sample book.

    Cheers
    Attached Files Attached Files

  6. #6
    Valued Forum Contributor tehneXus's Avatar
    Join Date
    04-12-2013
    Location
    Hamburg, Germany
    MS-Off Ver
    Work: MS-Office 2010 32bit @ Win8 32bit / Home: MS-Office 2016 32bit @ Win10 64bit
    Posts
    944

    Re: Which is the quickest way to loop through large data sets

    try this, runs 0,3 seconds on my machine: Loop through Cells.xlsm

    it produces a lot of zeros because of the date restriction, I checked the result only for the 1st entry in Vehicles

  7. #7
    Forum Contributor
    Join Date
    07-23-2012
    Location
    London, England
    MS-Off Ver
    Excel 2007
    Posts
    211

    Re: Which is the quickest way to loop through large data sets

    WOW tehneXus!!!

    It took 23minutes for my code to finish!!! Thanks so much, friend. Saved me days of searching.


+ Reply to Thread

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts

Search Engine Friendly URLs by vBSEO 3.6.0 RC 1