Skip to content

Instantly share code, notes, and snippets.

@valentinitnelav

valentinitnelav/closest_geodesic_distance_parallel_foreach_example.R

Last active August 29, 2017 12:38
Show Gist options
  • Save valentinitnelav/5759083f60eb7a7141f32466fe94d17a to your computer and use it in GitHub Desktop.
Save valentinitnelav/5759083f60eb7a7141f32466fe94d17a to your computer and use it in GitHub Desktop.
Download ZIP

Revisions

  1. valentinitnelav revised this gist Aug 29, 2017. 1 changed file with 14 additions and 5 deletions.
    19 changes: 14 additions & 5 deletions closest_geodesic_distance_parallel_foreach_example.R
    View file
    Original file line number Diff line number Diff line change
    @@ -1,10 +1,17 @@
    # Example of using foreach parallel loop
    # This should work both on Linux and Windows without any modifications.

    # for an interesting intro, check:
    # http://bgc.yale.edu/sites/default/files/ParallelR.html

    library(foreach)
    library(doParallel)

    FractionCores=0.7
    numCores <- round(FractionCores*detectCores())
    cl <- makeCluster(numCores)
    registerDoParallel(cl)
    # Prepare cluster
    FractionCores=0.7 # fraction of cores to use
    numCores <- round(FractCores * parallel::detectCores())
    cl <- parallel::makeCluster(numCores)
    doParallel::registerDoParallel(cl)

    system.time({
    res <-
    @@ -19,8 +26,10 @@ system.time({
    geosphere::dist2Line(p = PLdt_SPdf_NA[i,],
    line = poly.sp.rich,
    distfun = distHaversine)
    # if you have multiple operations here, it will return only the last assigned variable
    }
    })
    stopCluster(cl)

    # Note the fact that the foreach() loop returns object(s), so assign the foreach to a variable name like res <- foreach(...), otherwise you'll lost the results
    # Note the fact that the foreach() loop returns object(s), so assign the foreach to a variable name like res <- foreach(...),
    # otherwise you'll lost the results
  2. valentinitnelav created this gist Aug 29, 2017.
    26 changes: 26 additions & 0 deletions closest_geodesic_distance_parallel_foreach_example.R
    View file
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,26 @@
    library(foreach)
    library(doParallel)

    FractionCores=0.7
    numCores <- round(FractionCores*detectCores())
    cl <- makeCluster(numCores)
    registerDoParallel(cl)

    system.time({
    res <-
    foreach(
    i = 1:length(PLdt_SPdf_NA),
    .combine = rbind, # how to combine results (the default is a list)
    .multicombine = TRUE, # if .combine = rbind (above) then .multicombine defaults to TRUE anyway (so redundant here)
    .packages = c("geosphere"), # packages to load during parallel execution. This should be done even if you call geosphere::dist2Line below
    .options.multicore = list(preschedule=FALSE) # load balancing (only if needed, default is TRUE)
    ) %dopar%
    {
    geosphere::dist2Line(p = PLdt_SPdf_NA[i,],
    line = poly.sp.rich,
    distfun = distHaversine)
    }
    })
    stopCluster(cl)

    # Note the fact that the foreach() loop returns object(s), so assign the foreach to a variable name like res <- foreach(...), otherwise you'll lost the results