Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unexpected Behavior: error is not triggered #317

Closed
llrs opened this issue Jan 12, 2021 · 5 comments · Fixed by #318
Closed

Unexpected Behavior: error is not triggered #317

llrs opened this issue Jan 12, 2021 · 5 comments · Fixed by #318
Assignees

Comments

@llrs
Copy link

llrs commented Jan 12, 2021

Thank you for taking the time to maintain and reply to users of this package!

Troubleshooter: I am not sure all the examples of the API playground work with all the options. But I assume they do, as several of them I tried did run.

Describe the behavior:

I am testing how to add data o the database through the API with REDCapR

library("REDCapR")
url <- "https://redca.website.org"
token  <- Sys.getenv("REDCAP")

data <- data.frame(patient = paste0("BCN 00", 5:9), record_id = 18:22,
                   redcap_event_name = "week_0_arm_1")
validate_for_write(data)
da2 <- redcap_write(ds_to_write = data, redcap_uri = url, token = token)

Due to some protections on the website I think I cannot write (This is not the issue with REDCapR but it is related). However, I don't get any error message or warning that this couldn't be done, and instead da2$affected_ids contains the content of the website.
If this is expected to be a number then I think it should check that it is not a number and return an informative error (the da2$status_code is 200 so it is not enough to check the response code ), something along the lines:

The action couldn't be performed, please check the response below:

Expected behavior: An error or an informative message of the failing nature of the operation

Desktop (please complete the following information):

  • OS: Ubuntu 20.10
  • REDCap version REDCap 10.5.0
  • REDCapR Version 0.11.0

Additional context: First time we are using REDCap on my institution (and I am just an advanced R user not the REDCap admin).

@wibeasley
Copy link
Member

TLDR: I think an error is thrown, unless you have continue_on_error = TRUE. When this is true, a REDCapR bug is exposed, and I think this can be corrected bychanging an 'or' to 'and' when combining batches.

@llrs, Thanks for all the inside & advanced information. That helps a lot. Let me slightly restate some things to make sure I understand the problem, your suggestions, and how to reproduce it for testing. I'm getting slightly different results from you, so I want to talk through it. Did you have continue_on_error = TRUE on your computer, but not in the example pasted above? If so, I think explains the differences I see.

  1. Intentionally try to import/write a dataset to REDCap using a token that does NOT have "API Import/Update" privileges
    image

  2. REDCapR should recognize the import failed and return an error to the user

  3. Verify that the returned success is false and affected_ids vector is empty.

@llrs and others, do you agree with these steps? Do you like the returned values below? (Except for success in the third example.)

If you want to replicate the steps below, use this line of code. (The dataset doesn't conform, but it that's irrelevant, because the privileges errors are always triggered first.)

REDCapR::redcap_write(ds=mtcars, redcap_uri="https://bbmc.ouhsc.edu/redcap/api/", token="9A81268476645C4E5F03428B8AC3AA7B")

Here's what redcap_write() returns. (Which is what's in your example.)

> REDCapR::redcap_write(ds=mtcars, redcap_uri=credential$redcap_uri, token=credential$token)
Starting to update 32 records to be written at 2021-01-12 14:54:45.
Writing batch 1 of 1, with indices 1 through 32.
The REDCapR write/import operation was not successful.  The error message was:
ERROR: You do not have API Import/Update privileges
Error in REDCapR::redcap_write(ds = mtcars, redcap_uri = credential$redcap_uri,  : 
  The `redcap_write()` call failed on iteration 1. 

Here's what redcap_write_oneshot() returns:

> REDCapR::redcap_write_oneshot(ds=returned_object1$data, redcap_uri=credential$redcap_uri, token=credential$token)
The REDCapR write/import operation was not successful.  The error message was:
ERROR: You do not have API Import/Update privileges
$success
[1] FALSE

$status_code
[1] 403

$outcome_message
[1] "The REDCapR write/import operation was not successful.  The error message was:\nERROR: You do not have API Import/Update privileges"

$records_affected_count
[1] NA

$affected_ids
character(0)

$elapsed_seconds
[1] 0.1930091

$raw_text
[1] "ERROR: You do not have API Import/Update privileges"

Here's redcap_write() again, but with continue_on_error = TRUE.

> REDCapR::redcap_write(ds=mtcars, redcap_uri=credential$redcap_uri, token=credential$token, continue_on_error=TRUE)
Starting to update 32 records to be written at 2021-01-12 14:59:14.
Writing batch 1 of 1, with indices 1 through 32.
The REDCapR write/import operation was not successful.  The error message was:
ERROR: You do not have API Import/Update privileges
$success
[1] TRUE          # <--- THIS IS WRONG and should be fixed.  I'm changing the 'or' to 'and'  so `FALSE` is returned.  See the next commit

$status_code
[1] "403"

$outcome_message
[1] "The REDCapR write/import operation was not successful.  The error message was:\nERROR: You do not have API Import/Update privileges"

$records_affected_count
[1] 0

$affected_ids
character(0)

$elapsed_seconds
[1] 0.80884

Warning message:
In REDCapR::redcap_write(ds = mtcars, redcap_uri = credential$redcap_uri,  :
  The `redcap_write()` call failed on iteration 1. 

REDCapR 0.11.0.9002; REDCap 10.5.1

wibeasley added a commit that referenced this issue Jan 12, 2021
@llrs
Copy link
Author

llrs commented Jan 13, 2021

I didn't have the continue_on_error = TRUE set on my computer (but maybe that changed between 0.11.0 and the development version?). This might also be related to the protocol used by httr I think it is not the same on Linux than other OS (Are you using Windows?)

I can't generate a different API for the same project. I can only delete or regenerate it (and I won't do that because it takes time to get the token approved) and I can't toggle privileges:
Screenshot from 2021-01-13 09-20-14

I tried to reproduce this from another internet connection and didn't get the same result (with the same exact code used), now on affected_id I see:

library("REDCapR")
url <- "https://redcap.clinic.cat/redcap_v10.0.0/API/"
token  <- Sys.getenv("REDCAP_BARCELONA")

data <- data.frame(patient = paste0("BCN 00", 5:9), record_id = 18:22,
                   redcap_event_name = "week_0_arm_1")
validate_for_write(data)
#> # A tibble: 0 x 4
#> # … with 4 variables: field_name <chr>, field_index <int>, concern <chr>,
#> #   suggestion <chr>
da2 <- redcap_write(ds_to_write = data, redcap_uri = url, token = token)
#> Starting to update 5 records to be written at 2021-01-13 09:34:05.
#> Writing batch 1 of 1, with indices 1 through 5.
#> 13 records were written to REDCap in 0.2 seconds.
da2$affected_ids # Error
#>  [1] "\t\t\t\t\t\t<form action=\"/redcap_v10.5.0/API/\" method=\"post\" name=\"form\" enctype=\"multipart/form-data\">"                                                                                                                                                                                                                                                           
#>  [2] "\t\t\t\t\t\t\t<input type='hidden' name=\"token\" value=\"MYBARETOKEN!!\"><input type='hidden' name=\"content\" value=\"record\"><input type='hidden' name=\"format\" value=\"csv\"><input type='hidden' name=\"type\" value=\"flat\"><input type='hidden' name=\"data\" value=\"&quot;patient&quot;,&quot;record_id&quot;,&quot;redcap_event_name&quot;"
#>  [3] "&quot;BCN 005&quot;,18,&quot;week_0_arm_1&quot;"                                                                                                                                                                                                                                                                                                                            
#>  [4] "&quot;BCN 006&quot;,19,&quot;week_0_arm_1&quot;"                                                                                                                                                                                                                                                                                                                            
#>  [5] "&quot;BCN 007&quot;,20,&quot;week_0_arm_1&quot;"                                                                                                                                                                                                                                                                                                                            
#>  [6] "&quot;BCN 008&quot;,21,&quot;week_0_arm_1&quot;"                                                                                                                                                                                                                                                                                                                            
#>  [7] "&quot;BCN 009&quot;,22,&quot;week_0_arm_1&quot;\"><input type='hidden' name=\"overwriteBehavior\" value=\"overwrite\"><input type='hidden' name=\"returnContent\" value=\"ids\"><input type='hidden' name=\"returnFormat\" value=\"csv\">\t\t\t\t\t\t</form>"                                                                                                               
#>  [8] "\t\t\t\t\t\t<script type='text/javascript'>"                                                                                                                                                                                                                                                                                                                                
#>  [9] "\t\t\t\t\t\tdocument.form.submit();"                                                                                                                                                                                                                                                                                                                                        
#> [10] "\t\t\t\t\t\t</script>"                                                                                                                                                                                                                                                                                                                                                      
#> [11] "\t\t\t\t\t\t</body>"                                                                                                                                                                                                                                                                                                                                                        
#> [12] "\t\t\t\t\t\t</html>"                                                                                                                                                                                                                                                                                                                                                        
#> [13] "\t\t\t\t\t\t"

Created on 2021-01-13 by the reprex package (v0.3.0)

I can reproduce your example:

library("REDCapR")
redcap_write(ds=mtcars, redcap_uri="https://bbmc.ouhsc.edu/redcap/api/", 
             token="9A81268476645C4E5F03428B8AC3AA7B")
#> Starting to update 32 records to be written at 2021-01-13 09:33:15.
#> Writing batch 1 of 1, with indices 1 through 32.
#> The REDCapR write/import operation was not successful.  The error message was:
#> ERROR: You do not have API Import/Update privileges
#> Error in redcap_write(ds = mtcars, redcap_uri = "https://bbmc.ouhsc.edu/redcap/api/", : The `redcap_write()` call failed on iteration 1.

Created on 2021-01-13 by the reprex package (v0.3.0)

I think that I found the error:

Previously there was some documentation that pointed out to this url:
https://redcap.myorg.org/redcap_v10.0.0/API/ when I switched to https://redcap.myorg.org/api/ it works as expected for a token without Import/Update privileges:

The REDCapR write/import operation was not successful.  The error message was:
ERROR: You do not have API Import/Update privileges
Error in redcap_write(ds_to_write = data, redcap_uri = url, token = token) : 
  The `redcap_write()` call failed on iteration 1.  

Is there a way to check that I'm pointing to a valid API url or that I have the right permissions?

@llrs llrs closed this as completed Jan 13, 2021
@wibeasley
Copy link
Member

Weird, I wouldn't have expected that including the "version folder" would lead to different behavior like this.

This kinda suggests something changed between REDCap version 10.0.0 (ie the version in the url) and 10.5.0 (ie, the version you're currently running). I looked through the change log between 10.0.0 and 10.5.0, but didn't see anything suggesting their code changed related to this behavior.

The bare token concerns me too. I can't find any post about in in the forums. But I reported your experience in the bug reports. If you have access to the forums, please provide any more info you'd like.

https://community.projectredcap.org/questions/98856/bare-token-revealed-in-text-following-import-error.html
image

By the way, I don't think this is related to the OS of the client. I did test it on my Windows desktop, linux laptop, and Linux on Travis.

Thanks providing such good information. Tell me if there's something else.

@llrs
Copy link
Author

llrs commented Jan 13, 2021

I wasn't aware that there could be multiple API versions available. Maybe the problem is that there is a mismatch between the url and the version available?

Unfortunately I'm not able to register on the community site as I am not the admin of the REDCap at my institution.
Thanks for reporting the token leak to the project community.

If I find something else using the package I'll get in touch.

@wibeasley wibeasley self-assigned this Jan 13, 2021
wibeasley added a commit that referenced this issue Jan 13, 2021
reference the bug found by @llrs in #317
@wibeasley wibeasley mentioned this issue Jan 13, 2021
Merged
@wibeasley
Copy link
Member

@llrs, Rob Taylor is looking into this, but needs some more info to replicate the problem. If you email me your contact info (at [email protected]), I'll show you want he's looking at, and connect you.

Alternatively, you REDCap site admin can participate in the post. Again, I regret that AnswerHub is so closed off from most API developers. https://community.projectredcap.org/questions/98856/bare-token-revealed-in-text-following-import-error.html

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants