#' ---
#' title: "Chemical Structure Inputs for PUG-REST"
#' author: "S. Kim, J. Cuadros"
#' date: "August 21st, 2019"
#' output: html_document
#' editor_options: 
#'   chunk_output_type: console
#' ---
#' 
## ----setup, include=FALSE------------------------------------------------
knitr::opts_chunk$set(echo = TRUE)

#' 
#' ## Objectives
#' 
#' - Use SMILES and InChI strings to specify the input compound for a PUG-REST request.
#' - Use a structure-data (SD) file to specify the input compound for a PUG-REST request.
#' - Learn to submit a PUG-REST request using the HTTP-POST method.
#' 
#' 
#' ## Background
#' 
#' You can use a chemical structure as an input for a PUG-REST request.  PUG-REST accepts some popular chemical structure line notations such as SMILES and InChI strings.  It is also possible to use an Structure-Data File (SDF) as a structure input.<br><br>
#' To learn how to specify the structure input in a PUG-REST request, one needs to know that there are two methods by which data are transferred from clients (users) and servers (PubChem) through PUG-REST.  Discussing what these methods are in detail is beyond the scope of this material, and it is enough to know three things:
#' 
#' - When you make a PUG-REST request by typing the request URL in the address bar of your web browser (such as Google Chrome, MS Internet Explorer), the HTTP GET method is used
#' - The HTTP GET method transfers information encoded in a single-line URL.
#' - Some chemical structure inputs are not appropriate to encode in a single-line URL (because they may contain special characters not compatible with the URL syntax, span over multiple lines, or too long), and the HTTP POST needs to be used for such cases.
#' 
#' For more information on HTTP GET and POST, read the following documents.
#' 
#' - HTTP request methods (https://www.w3schools.com/tags/ref_httpmethods.asp)
#' - GET vs. POST (https://www.diffen.com/difference/GET-vs-POST-HTTP-Requests)
#' 
#' 
#' Let's start checking if the ```httr``` package is available, installing it if needed. Then, we load it.
#' 
#' 
## ------------------------------------------------------------------------
if(!require("httr", quietly=TRUE)) {
  install.packages("httr", repos="https://cloud.r-project.org/",
         quiet=TRUE, type="binary")
  library("httr", quietly=TRUE)
}

#' 
#' Packages are the way that libraries (additional functions, data types, constants, data sets...) are distributed in the R environment. In order to use a package, it has to be installed (only once per running environment) with the  ```install.packages``` function. Then, if we load it (in a specific R session) using ```library```, its functions can be called as they were in the base environment.
#' 
#' The ```require``` function checks whether a package has already been installed and loads it if so. It returns a logical value than can be used to install the package if it was not available.
#' 
#' The ```httr``` package allows a finer grade manipulation of the HTTP communications. That's why we will use it in this activity. A quick introduction to the package is available at https://cran.r-project.org/web/packages/httr/vignettes/quickstart.html.
#' 
#' 
#' ## 1. Using the HTTP GET method.
#' 
#' ### 1.1. Structure encoded in the URL path.
#' 
#' In some cases, you can encode a chemical structure in the PUG-REST request URL path as in the following example.
#' 
#' 
## ------------------------------------------------------------------------
prolog <- 'https://pubchem.ncbi.nlm.nih.gov/rest/pug'
smiles1 <- "CC(C)CC1=CC=C(C=C1)C(C)C(=O)O"
url <- paste(prolog, "/compound/smiles/", smiles1, "/cids/txt", sep="")
url
res = GET(url)
content(res,"text",encoding="UTF-8")

#' 
#' 
#' It should be noteworthy that some SMILES strings contain special characters, such as the forward slash ("/"), which is also used in the URL path.  These special characters conflict with the PUG-REST request URL syntax, causing an error when used in the PUG-REST request URL.
#' 
#' Also note that the backslash character has to be escaped when used in a string in R. To include a blackslash, we have to write ```\\```.
#' 
## ------------------------------------------------------------------------
smiles2 <- "CC1=C([C@@](SC1=O)(C)/C=C(\\C)/C=C)O"
url <- paste(prolog, "/compound/smiles/", smiles2, "/cids/txt", sep="")
url
res <- GET(url)
content(res,"text",encoding="UTF-8")

#' 
#' 
#' ### 1.2. Structure encoded as a URL argument
#' 
#' To circumvent the issue mentioned above, the SMILES string may be encoded as the URL argments (as an optional parameter followed by the "?" character).
#' 
#' 
## ------------------------------------------------------------------------
url2 <- paste(prolog, "/compound/smiles/cids/txt?smiles=", smiles2, sep="")
url2
res2 <- GET(url2)
content(res2,"text",encoding="UTF-8")

#' 
#' 
#' ### 1.3. Structure encoded as a URL-encoded URL argument
#' 
#' It is also possible to pass the structure query as a URL-encoded argument.  The following example does the same task as the previous example does.
#' 
#' This is safer in case the argument includes ```&```, ```?``` or other reserved characters.
#' 
#' 
## ------------------------------------------------------------------------
url3 <- paste(prolog, "/compound/smiles/cids/txt?smiles=", URLencode(smiles2,reserved=TRUE), sep="")
url3
res3 <- GET(url3)
content(res3,"text",encoding="UTF-8")

#' 
#' 
#' The object returned from a web service request (res, res2, and res3 in our examples) contains information on the request URL through which the data have been retrieved.  This information can be accessed using the ```$url``` attribute of the object, as shown in this example:
#' 
#' 
## ------------------------------------------------------------------------
res2$url   # from (2) structure encoded as a URL argument
res3$url   # from (3) structure encoded as a URL-encoded URL argument

#' 
#' Note that URL-encoding does not work for PubChem when including structure information in the URL path.
#' 
#' 
#' **Exercise 1a:** Retrieve the hydrogen bond donor and acceptor counts, TPSA, and XLogP of the chemical represented by the SMILES string: "C1=CC(=C(C=C1Cl)O)OC2=C(C=C(C=C2)Cl)Cl".  When construct a PUG-REST url for this request, encode the structure in the URL path.
#' 
#' 
## ------------------------------------------------------------------------
# Write your code here



#' 
#' 
#' **Exercise 1b:** Get the CID corresponding to the following InChI string, using the HTTP GET method.
#' 
#' 
## ------------------------------------------------------------------------
inchi <- "InChI=1S/C17H14O4S/c1-22(19,20)14-9-7-12(8-10-14)15-11-21-17(18)16(15)13-5-3-2-4-6-13/h2-10H,11H2,1H3"

# Write your code here



#' 
#' 
#' ## 2. Using the HTTP POST method
#' 
#' ### 2.1. Comparison of HTTP POST and GET
#' 
#' All the three examples above use the HTTP GET method, as implied in the use of the ```GET``` function.  Alternatively, one can use the HTTP POST method.  For example, the following example returns the identical result as the last two HTTP GET examples.
#' 
#' 
## ------------------------------------------------------------------------
url <- paste(prolog, "/compound/smiles/cids/txt", sep="")
struct <- list(smiles=smiles2)

res <- POST(url, body = struct)
res$url
content(res,"text",encoding="UTF-8")


#' 
#' 
#' When using the HTTP POST method, information is passed as data through the body argument.  Because of this, the URL stored in the ```res$url``` does not contain structure informaion.
#' 
#' 
#' ### 2.2. HTTP POST for multi-line structure input
#' 
#' The HTTP POST method should be used if the input molecular structure for PUG-REST request span over multiple lines (e.g., stored in a structure-data file (SDF) format).  The SDF file contains structure information of a molecule in a multi-line format, along with other data.
#' 
#' 
## ------------------------------------------------------------------------
mysdf <- "1983
  -OEChem-07241917072D

 20 20  0     0  0  0  0  0  0999 V2000
    2.8660   -2.5950    0.0000 O   0  0  0  0  0  0  0  0  0  0  0  0
    4.5981    1.4050    0.0000 O   0  0  0  0  0  0  0  0  0  0  0  0
    2.8660    1.4050    0.0000 N   0  0  0  0  0  0  0  0  0  0  0  0
    2.8660    0.4050    0.0000 C   0  0  0  0  0  0  0  0  0  0  0  0
    3.7320   -0.0950    0.0000 C   0  0  0  0  0  0  0  0  0  0  0  0
    2.0000   -0.0950    0.0000 C   0  0  0  0  0  0  0  0  0  0  0  0
    3.7320   -1.0950    0.0000 C   0  0  0  0  0  0  0  0  0  0  0  0
    2.0000   -1.0950    0.0000 C   0  0  0  0  0  0  0  0  0  0  0  0
    2.8660   -1.5950    0.0000 C   0  0  0  0  0  0  0  0  0  0  0  0
    3.7320    1.9050    0.0000 C   0  0  0  0  0  0  0  0  0  0  0  0
    3.7320    2.9050    0.0000 C   0  0  0  0  0  0  0  0  0  0  0  0
    4.2690    0.2150    0.0000 H   0  0  0  0  0  0  0  0  0  0  0  0
    1.4631    0.2150    0.0000 H   0  0  0  0  0  0  0  0  0  0  0  0
    2.3291    1.7150    0.0000 H   0  0  0  0  0  0  0  0  0  0  0  0
    4.2690   -1.4050    0.0000 H   0  0  0  0  0  0  0  0  0  0  0  0
    1.4631   -1.4050    0.0000 H   0  0  0  0  0  0  0  0  0  0  0  0
    4.3520    2.9050    0.0000 H   0  0  0  0  0  0  0  0  0  0  0  0
    3.7320    3.5250    0.0000 H   0  0  0  0  0  0  0  0  0  0  0  0
    3.1120    2.9050    0.0000 H   0  0  0  0  0  0  0  0  0  0  0  0
    2.3291   -2.9050    0.0000 H   0  0  0  0  0  0  0  0  0  0  0  0
  1  9  1  0  0  0  0
  1 20  1  0  0  0  0
  2 10  2  0  0  0  0
  3  4  1  0  0  0  0
  3 10  1  0  0  0  0
  3 14  1  0  0  0  0
  4  5  2  0  0  0  0
  4  6  1  0  0  0  0
  5  7  1  0  0  0  0
  5 12  1  0  0  0  0
  6  8  2  0  0  0  0
  6 13  1  0  0  0  0
  7  9  2  0  0  0  0
  7 15  1  0  0  0  0
  8  9  1  0  0  0  0
  8 16  1  0  0  0  0
 10 11  1  0  0  0  0
 11 17  1  0  0  0  0
 11 18  1  0  0  0  0
 11 19  1  0  0  0  0
M  END
> <PUBCHEM_COMPOUND_CID>
1983

> <PUBCHEM_COMPOUND_CANONICALIZED>
1

> <PUBCHEM_CACTVS_COMPLEXITY>
139

> <PUBCHEM_CACTVS_HBOND_ACCEPTOR>
2

> <PUBCHEM_CACTVS_HBOND_DONOR>
2

> <PUBCHEM_CACTVS_ROTATABLE_BOND>
1
$$$$
"

#' 
#' Multi-line string in R can be written without any special consideration.  This multi-line sdf data is used as an input for a PUG-REST request through the HTTP POST.
#' 
#' 
## ------------------------------------------------------------------------
url <- paste(prolog, "/compound/sdf/cids/txt", sep="")
mydata <- list(sdf=mysdf)

res <- POST(url, body = mydata)
res$url
content(res,"text",encoding="UTF-8")

#' 
#' Note that HTTP POST should be used for the input specification using a SDF format. Although HTTP GET may work if data is URL-encoded, it will be more dependent of URL length limitations.
#' 
#' 
## ------------------------------------------------------------------------
url3 <- paste(prolog, "/compound/sdf/cids/txt?sdf=", URLencode(mysdf,reserved=TRUE), sep="")
url3
res3 <- GET(url3)
content(res3,"text",encoding="UTF-8")

#' 
#' 
#' ## 2.3. HTTP POST for multi-line structure input
#' 
#' One may want to use the structure stored in a file as the input for a PUG-REST request.  The following code shows how to read an SDF file into a variable.
#' 
#' 
## ------------------------------------------------------------------------
mysdf <- paste(readLines('Structure2D_CID_5288826.sdf'),collapse="\n")
cat(mysdf)

#' 
#' ```cat``` is used in this example instead of ```print``` (which can be omitted) as it produces a nicer print-out for multiple-line strings.
#' 
#' 
#' Now the structure stored in the "mysdf" can be used in a PUG-REST request through HTTP-POST.  For example, the code cell below shows how to retrieve various names (also called "synonyms") of the input structure.
#' 
#' 
## ------------------------------------------------------------------------
url <- paste(prolog, "/compound/sdf/synonyms/txt", sep="")
mydata <- list(sdf=mysdf)

res <- POST(url, body = mydata)
res$url
cat(content(res,"text",encoding="UTF-8"))

#' 
#' 
#' **Exercise 2a:** Retrieve (in the **CSV** format) the XlogP, molecular weight, hydrogen bond donor count, hydrogen bond aceptor count, and TPSA of the compounds contained in the five sdf files (link to the data files).
#' 
#' - Use a for loop to retrieve the data for each compound.
#' - Remember to add some sleep time (e.g 0.5 seconds) after retrieving the data for each compound.
#' - Refer to the __lecture 1__ notebook to see how to merge the multiple CSV outputs into a single data frame.
#' 
#' 
## ------------------------------------------------------------------------
files <- c('l02_ex2b_compound1.sdf','l02_ex2b_compound2.sdf','l02_ex2b_compound3.sdf',
         'l02_ex2b_compound4.sdf','l02_ex2b_compound5.sdf')

# Write your code here



#' 
#'