Apply a function to batches of data in an Andromeda table

Source: R/Operations.R

Apply a function to batches of data in an Andromeda table

batchApply(tbl, fun, ..., batchSize = 1e+05, progressBar = FALSE, safe = FALSE)

Arguments

tbl

An Andromeda table (or any other 'DBI' table).

fun

A function where the first argument is a data frame.

...

Additional parameters passed to fun.

batchSize

Number of rows to fetch at a time.

progressBar

Show a progress bar?

safe

Create a copy of tbl first? Allows writing to the same Andromeda as being read from.

Value

Invisibly returns a list of objects, where each object is the output of the user-supplied function applied to a batch

Details

This function is similar to the lapply() function, in that it applies a function to sets of data. In this case, the data is batches of data from an Andromeda table. Each batch will be presented to the function as a data frame.

See also

groupApply()

Examples

andr <- andromeda(cars = cars)

fun <- function(x) {
  return(nrow(x))
}

result <- batchApply(andr$cars, fun, batchSize = 25)

result
#> [[1]]
#> [1] 25
#> 
#> [[2]]
#> [1] 25
#> 
# [[1]] 
# [1] 25 
# 
# [[2]] 
# [1] 25

close(andr)