Arch Linux User Repository

Hopefully c-ares that will get sorted out soon. Also I had to add python-pytz for the check to work, otherwise it would error out and not finish the build. You may need to add that as a check dependency.

I guess technically c-ares-cmake is just makedepends and c-ares is depends. Ideally c-ares starts shipping the cmake file which would make things easier. There is a bug report for it: https://bugs.archlinux.org/task/61918?project=1&string=c-ares

is c-ares-cmake a dependency or a build dependency? The reason I ask is that cmake is a build dependency not a regular dependency so I'm not sure why the cmake variant of c-ares would be needed beyond the build.

I think the cmake.patch file may need to be updated: it appears to point to 1.0.0 files and fails on my system.

@thrasibule That was the case. I was, however, able to get everything working by just sticking to the static libraries (I still had to replace zstd-static with zstd before building arrow though). Everything appears to be working smoothly now though! I appreciate you taking the time to update the package and respond to my messages. Hopefully things will stabilize some down the road as the package matures and you won't have to go through so much effort :)

That's strange. So this is with -DARROW_ORC=OFF and -DARROW_WITH_SNAPPY=ON? I tried on my machine and it works fine, not sure what could be the cause.

Thanks for the quick response and suggestions! I tried first with just disabling the ORC tests and it was able to get quite close:

94% tests passed, 3 tests failed out of 49

Label Time Summary:
arrow-tests           =   9.85 sec*proc (29 tests)
arrow_compute         =   0.83 sec*proc (4 tests)
arrow_dataset         =   0.77 sec*proc (9 tests)
arrow_python-tests    =   0.41 sec*proc (1 test)
filesystem            =   0.61 sec*proc (2 tests)
parquet-tests         =   3.82 sec*proc (6 tests)
unittest              =  15.69 sec*proc (49 tests)

Total Test time (real) =  15.78 sec

The following tests FAILED:
     44 - parquet-internals-test (Failed)
     45 - parquet-reader-test (Failed)
     47 - parquet-arrow-test (Failed)

From the test logs, the failures all appear to be related to missing Snappy support, e.g.:

unknown file: Failure
C++ exception with description "NotImplemented: Snappy codec support not built" thrown in the test body.
[  FAILED  ] TestDumpWithLocalFile.DumpOutput (1 ms)
[ RUN      ] TestStatisticsSortOrder/0.MinMax
...
unknown file: Failure
C++ exception with description "NotImplemented: Snappy codec support not built" thrown in the test body.
[  FAILED  ] TestStatisticsSortOrder/1.MinMax, where TypeParam = parquet::PhysicalType<(parquet::Type::type)2> (0 ms)
[----------] 1 test from TestStatisticsSortOrder/1 (0 ms total)

Snappy appeared to be detected fine though:

-- Building using CMake version: 3.18.1
-- Arrow version: 1.0.0 (full: '1.0.0')
-- Arrow SO version: 100 (full: 100.0.0)
ARROW_SNAPPY_BUILD_VERSION: 1.1.8

Possibly related: https://issues.apache.org/jira/browse/PARQUET-1856# (similar failures, even when snappy tests are set to be skipped.. )

The issue is not plasma not flight but orc. Orc needs to be compiled statically, and if it's compiled with a different version of protobuf, there are missing references. So I would either recompile and install apache-orc (which depends on the static versions of protobuf, zstd, lz4 and snappy). Then reinstall the non static versions of protobuf, zstd and lz4 (which is fine since they were statically compiled in apache-orc). Then arrow should compile fine. Alternatively, you can disable orc (-DARROW_ORC=OFF), and it should compile fine too.

Thanks for updating the package to 1.0! Still running into compilation issues, most recently with the arrow-misc-test:

[ 45%] Linking CXX executable ../../release/arrow-misc-test
/usr/bin/ld: ../../release/libarrow.so.100.0.0: undefined reference to `google::protobuf::internal::UnknownFieldParse(unsigned int, google::protobuf::internal::InternalMetadataWithArena*, char const*, google::protobuf::internal::ParseContext*)'
/usr/bin/ld: ../../release/libarrow.so.100.0.0: undefined reference to `google::protobuf::internal::AssignDescriptors(google::protobuf::internal::DescriptorTable const*)'
/usr/bin/ld: ../../release/libarrow.so.100.0.0: undefined reference to `google::protobuf::internal::ComputeUnknownFieldsSize(google::protobuf::internal::InternalMetadataWithArena const&, unsigned long, google::protobuf::internal::CachedSize*)'
collect2: error: ld returned 1 exit status
make[2]: *** [src/arrow/CMakeFiles/arrow-misc-test.dir/build.make:178: release/arrow-misc-test] Error 1
make[1]: *** [CMakeFiles/Makefile2:1358: src/arrow/CMakeFiles/arrow-misc-test.dir/all] Error 2
make: *** [Makefile:160: all] Error 2
==> ERROR: A failure occurred in build().
    Aborting...

In order to get that far, I first had to:

1. uninstall zstd-static, which was causing issues with CMake's ability to detect zstd
2. disable "plasma" tests (-DARROW_PLASMA=OFF in the PKGBUILD)

Issues appear to still be due to incompatibility with protobuf. Disabling the "flight" tests doesn't help here though.

Reported upstream: https://issues.apache.org/jira/browse/ARROW-9025

For anyone else trying to install this to get the R package running, I fought with it a bit last night & this may help others:

Because grpc 1.28+ is currently causing issues (I was on 1.29.1-1), start with a:

yay --editmenu -S arrow

& edit the pkgbuild to turn -DARROW_FLIGHT to OFF (thanks @skogler). Also add any other flags here - I wanted -DARROW_WITH_SNAPPY=ON.

Then continue the install.

After a successful yay install, for the R package install you need to set the environment variable NOT_CRAN=true & because the CRAN version is currently ahead of AUR, you'll need to install the older CRAN version:

packageurl <- "http://cran.r-project.org/src/contrib/Archive/arrow/arrow_0.17.0.tar.gz"
install.packages(packageurl, repos=NULL, type="source")