Search Criteria
Package Details: apache-spark 3.0.1-0
Git Clone URL: | https://aur.archlinux.org/apache-spark.git (read-only, click to copy) |
---|---|
Package Base: | apache-spark |
Description: | Apache Spark is a unified analytics engine for large-scale data processing. |
Upstream URL: | http://spark.apache.org |
Keywords: | spark |
Licenses: | |
Submitter: | huitseeker |
Maintainer: | lukaszimmermann (emanuelfontelles) |
Last Packager: | emanuelfontelles |
Votes: | 49 |
Popularity: | 0.36 |
First Submitted: | 2015-10-04 09:31 |
Last Updated: | 2020-12-03 04:29 |
Dependencies (9)
- java-environment<=11 (jdk5, tuxjdk, jdk7-openjdk-infinality, java-8-openjdk-shenandoah, jdk8-openjdk-infinality, java-gcj-compat, jdk9, zulu-10-bin, jdk-devel, jdk8-arm64, jdk6, jdk12-openj9-bin, zulu-12-bin, jdk12, jdk8-openjdk-dcevm, jdk11-graalvm-ee-bin, jdk8-arm, jdk13-openj9-bin, jdk9-openj9-bin, jdk10, jdk11-openjdk-dcevm, jdk8-openjdk-jetbrains, jdk8-openjdk-shenandoah, jdk10-openj9-bin, jdk14-openj9-bin, sapmachine-jdk-bin, jdk7, jdk8-j9-bin, java-openjdk-bin, jdk, jdk-openj9-bin, jdk-arm, jdk-adoptopenjdk, jdk11-adoptopenjdk, liberica-jdk-full-bin, jdk8-jetbrains, zulu-embedded-jdk8, amazon-corretto-11, liberica-jdk-8-full-bin, liberica-jdk-11-bin, liberica-jdk-11-full-bin, liberica-jdk-11-lite-bin, liberica-jre-11-bin, liberica-jre-11-full-bin, liberica-jre-8-full-bin, amazon-corretto-8, jdk8, jdk11, jdk8-graalvm-bin, jdk11-graalvm-bin, zulu-embedded-jdk, jdk7r1-j9-bin, jdk7-j9-bin, zulu-15-bin, zulu-14-bin, zulu-13-bin, jdk8-adoptopenjdk, zulu-embedded-jdk11, jdk-bcl, jdk8-openj9-bin, zulu-11-bin, zulu-8-bin, amazon-corretto-15, jdk-openj9, jre-jetbrains, java-openjdk-ea-bin, java-openjdk-loom-ea-bin, jdk-jetbrains, jdk11-openj9-bin, jdk11-jetbrains-imfix, jdk-openjdk, jdk11-openjdk, jdk7-openjdk, jdk8-openjdk)
- java-environment>=6 (jdk5, tuxjdk, jdk7-openjdk-infinality, java-8-openjdk-shenandoah, jdk8-openjdk-infinality, java-gcj-compat, jdk9, zulu-10-bin, jdk-devel, jdk8-arm64, jdk6, jdk12-openj9-bin, zulu-12-bin, jdk12, jdk8-openjdk-dcevm, jdk11-graalvm-ee-bin, jdk8-arm, jdk13-openj9-bin, jdk9-openj9-bin, jdk10, jdk11-openjdk-dcevm, jdk8-openjdk-jetbrains, jdk8-openjdk-shenandoah, jdk10-openj9-bin, jdk14-openj9-bin, sapmachine-jdk-bin, jdk7, jdk8-j9-bin, java-openjdk-bin, jdk, jdk-openj9-bin, jdk-arm, jdk-adoptopenjdk, jdk11-adoptopenjdk, liberica-jdk-full-bin, jdk8-jetbrains, zulu-embedded-jdk8, amazon-corretto-11, liberica-jdk-8-full-bin, liberica-jdk-11-bin, liberica-jdk-11-full-bin, liberica-jdk-11-lite-bin, liberica-jre-11-bin, liberica-jre-11-full-bin, liberica-jre-8-full-bin, amazon-corretto-8, jdk8, jdk11, jdk8-graalvm-bin, jdk11-graalvm-bin, zulu-embedded-jdk, jdk7r1-j9-bin, jdk7-j9-bin, zulu-15-bin, zulu-14-bin, zulu-13-bin, jdk8-adoptopenjdk, zulu-embedded-jdk11, jdk-bcl, jdk8-openj9-bin, zulu-11-bin, zulu-8-bin, amazon-corretto-15, jdk-openj9, jre-jetbrains, java-openjdk-ea-bin, java-openjdk-loom-ea-bin, jdk-jetbrains, jdk11-openj9-bin, jdk11-jetbrains-imfix, jdk-openjdk, jdk11-openjdk, jdk7-openjdk, jdk8-openjdk)
- hadoop (optional) – support for running on YARN
- ipython (optional) – ipython3 support for pyspark
- ipython2 (optional) – ipython2 support for pyspark
- python (python-dbg) (optional) – python3 support for pyspark
- python2 (optional) – python2 support for pyspark
- r (microsoft-r-open, r-mkl) (optional) – support for sparkR
- rsync (rsync-git) (optional) – support rsync hadoop binaries from master
Required by (1)
- polynote (optional)
Latest Comments
1 2 3 4 5 6 Next › Last »
dmfay commented on 2020-04-22 20:13
For 2.4.5 (also fixes the worker unit description):
ryukinix commented on 2020-01-17 23:12
Updating to spark 3.0.0-preview2 it will make works with Python3.8. I'm using this modified version of PKGBUILD: https://github.com/ryukinix/apache-spark-pkgbuild. Until now it's working fine.
5 days ago it was released the v2.4.5 github tag, but is not available yet on apache spark archive the compiled version, for that reason I used 3.0.0 preview2 version which is the most recent version available in the archive.
v2.4.5 and v3.0.0 versions contains the commit that fix the problems with python3.8: https://github.com/apache/spark/commit/811d563fbf60203377e8462e4fad271c1140b4fa
YaLTeR commented on 2019-12-13 08:25
Doesn't seem to work with Python 3 (the default)?
Update: looks like it's Python 3.8; installing python37 from the AUR and using that seems to work.
tir commented on 2019-07-17 10:35
I had success with the following PKGBUILD (patch: https://git.io/fj1CT).
There is another, unrelated issue probably with JLine (as noted here: https://github.com/sanori/spark-sbt/issues/4#issuecomment-401621777), for which the workaround would be:
lukaszimmermann commented on 2019-07-15 09:05
Hi, sorry for being unresponsive the last couple of month. Emanuel I added you as co-maintainer, can you perhaps update this package with the new PKGBUILD for Spark 2.4.3.
emanuelfontelles commented on 2019-07-13 17:56
Hey guys, here are a full PKGBUILD with Spark-2.4.3, with Hadoop binaries, its totally works
marcinn commented on 2019-06-14 20:46
PKGBUILD patch for 2.4.3:
sabitmaulanaa commented on 2019-05-27 14:23
is this package will be updated?
lukaszimmermann commented on 2019-04-26 09:29
Works, thanks!
blurbdust commented on 2019-04-26 00:31
Here is my current PKGBUILD that works for the latest spark as on 4/23/19
https://gist.github.com/blurbdust/3e03531d890c8bcf1da3c3d1192ce4d5