@ezraporter, good catch, and thanks for doing the hard work to figure it out. Do you want to take a shot at finishing it?

If so, here's a suggestion how to dynamically name & append those to post_body.

post_body <- list(token="abcd", content="metadata", returnFormat="csv")
forms     <- NULL                               # The default
# forms     <- c("first", "second", "third")      # Well-behaved
# forms     <- character(0)                       # Still legal

if (0L < length(forms)) {
  names(forms) <- sprintf("forms[%i]", seq_along(forms))
  # form_list <-
  #   c(
  #     "forms[0]" = "first",
  #     "forms[1]" = "second",
  #     "forms[2]" = "third"
  #   )
  
  post_body <-
    append(
      post_body,
      forms
    )
}
print(post_body)

@wibeasley yup I can take a shot at this. I'll make a PR

@ezraporter, it looks like you haven't tried it yet? Good, b/c I've changed my mind. I think it needs to be filtered in R --not in REDCap. In other words, everything gets extracted from REDCap and then dplyr::filter() removes the rows. Since this is metadata with at most 5k rows (and not real data with sometimes 100k rows), there shouldn't be much of a performance difference.

I think the metadata should always return the plumbing variables when they exist (ie, record_id, event_name, redcap_repeating_instrument, and redcap_repeating_instance). see #439

Yes, sorry! Last week got away from me. Glad I didn't hold you up. That approach makes sense to me. Still helpful for me to implement that in a PR using filter()? I have some time today so I can give it a shot

I did something differently this weekend so it's not as important, but I'm still leaning toward dplyr::filter() for two reasons

1. there might be an occasion where the funtion still needs to know about the other fields, even if they're not returned.
2. i'm kinda wondering if the API might change, since the way it accepts forms is different than how the other API functions accept forms (as well as records, events, & fields).

I guess performance is the only reason I'd say to filter at the server. Can you do a quick proof of concept?

For a gerbil, use the massive 35k column project (https://github.com/OuhscBbmc/REDCapR/blob/main/inst/misc/example.credentials#L26). If there isn't a noticeable performance difference, use dplyr::filter()?

Yes I'll give it a shot!

I'm actually seeing a non-trivial performance difference. Takes less than a second to pull metadata for a single form from the 35k field redcap but almost 20 seconds to get all the metadata. It's about 4 seconds to get all the metadata from a 5k field redcap.

devtools::load_all()

# 35k field redcap
redcap_metadata_read(
  "https://bbmc.ouhsc.edu/redcap/api/",
  "5C1526186C4D04AE0A0630743E69B53C"
)$elapsed_seconds

#> [1] 18.2162

# 5k field redcap
redcap_metadata_read(
  "https://bbmc.ouhsc.edu/redcap/api/",
  "1C31398F332FCACA4C0A7B93B18D5CD4"
)$elapsed_seconds

#> [1] 3.845438

# Single form for 35k field redcap
# Defined for testing here: https://github.com/ezraporter/REDCapR/commit/54012b611d444fefbd6df2945c91ccae386a8c17
redcap_metadata_read_hacky_single_form(
  "https://bbmc.ouhsc.edu/redcap/api/",
  "5C1526186C4D04AE0A0630743E69B53C",
  forms = "form_0001"
)$elapsed_seconds

#> [1] 0.511425

Given that, make sense to mock up a full-fledged version that filters at the server?

wow, I didn't expect that big of a difference. That crosses my threshold, and it sounds like it crosses your too. So yes, please filter at the server. Thanks for running those scenarios to inform our decision.