-
Notifications
You must be signed in to change notification settings - Fork 0
r-spark/sparkavroudf
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Folders and files
Name | Name | Last commit message | Last commit date | |
---|---|---|---|---|
Repository files navigation
--- title: "What the Package Does (Title Case)" output: github_document: fig_width: 9 fig_height: 5 --- ## Building First build this package, then build its jars by running: ```{r eval=FALSE} spec <- sparklyr::spark_compilation_spec(spark_version = "2.4", jar_name = "sparkavroudf-2.4-2.11.jar", jar_dep = normalizePath("deps/spark-avro_2.12-2.4.2.jar")) sparklyr::compile_package_jars(spec = spec) ``` then build the R package as usual. ## Getting Started Connect and test this package as follows: ```{r} library(sparkavroudf) library(sparklyr) sc <- spark_connect(master = "local", version = "2.4") sparklyr::invoke_static(sc, "sparkavroudf.AvroUtils", "toAvro", spark_dataframe(sdf_len(sc, 3)), invoke_new(sc, "org.apache.spark.sql.Column", "id")) %>% sdf_register() ``` ```{r} spark_disconnect_all() ```
About
Support for Avro with UDF-like interface
Resources
Stars
Watchers
Forks
Releases
No releases published
Packages 0
No packages published