Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

update instructions #14

Open
wants to merge 2 commits into
base: branch-1.10.0
Choose a base branch
from

Conversation

mitochon
Copy link

Issue #, if available:

Description of changes:

Outdated step

  • branch-2.3 of hive will cause issues due to API change here that will cause this error downstream
    error: AWSCatalogMetastoreClient is not abstract and does not override abstract method listPartitionValues(PartitionValuesRequest) in IMetaStoreClient

Missing step

  • Building hive 1.2 will put files into ~/.m2/repository/org/apache/hive/ ... whereas these need to be under ~/.m2/repository/org/spark-project/hive/

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

README.md Outdated
@@ -15,11 +15,11 @@ Obtain a copy of Hive from GitHub at https://github.com/apache/hive.

To build the Hive client, you need to first apply this [patch](https://issues.apache.org/jira/secure/attachment/12958418/HIVE-12679.branch-2.3.patch). Download this patch and move it to your local Hive git repository you created above. Apply the patch and build Hive.

git checkout branch-2.3
git checkout tags/rel/release-2.3.4 -branch rel-2.3.4

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should it be "-b"?
git checkout tags/rel/release-2.3.4 -b rel-2.3.4

Note: I am running git version 2.17.2 (Apple Git-113)

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, typo on my part. Thanks for the correction.

@ryan-carlson
Copy link

One other note:

I was having trouble building hive getting an error The following artifacts could not be resolved: javax.jms:jms:jar:1.1.

I found that I had to add this to the pom.xml file to get it to resolve. I took this suggestion from https://stackoverflow.com/questions/4908651/the-following-artifacts-could-not-be-resolved-javax-jmsjmsjar1-1.

    <repository>
      <id>repository.jboss.org-public</id>
      <name>JBoss.org Maven repository</name>
      <url>https://repository.jboss.org/nexus/content/groups/public</url>
    </repository>

@mitochon
Copy link
Author

mitochon commented Aug 7, 2019

The artifact is on maven central -> https://search.maven.org/artifact/javax.jms/jms/1.1/jar
I was able to run without error on my machine.

@BVemployee
Copy link

I had to add the hadoop-common dependency to the spark-hive-shims pom directly but otherwise this also built fine on my system. It would be great to get this merged in, I spent much too long fighting w/ the build instructions before looking this issue.

@gaurang101197
Copy link

gaurang101197 commented May 29, 2020

@mitochon I followed steps given at here. I am able to build client aws-glue-datacatalog-hive2-client, and I am not getting 2.3.4-SNAPSHOT as a release version while building hive, instead getting 2.3.4 version in my local m2 directory ~/.m2/repository/org/apache/hive/hive/2.3.4/. But when I run the hive server using the build generated with this patch, I followed steps mentioned at building hive from source to run hive server. I am not able to connect to glue, I have checked that instance has access to connect glue. And when I run the query like show databases and show tables, it is giving default as a only database and empty table lists. Can anyone please help ?

@mitochon
Copy link
Author

@gaurang101197 you should probably file an issue with more details of your situation

A few sanity checks to try

  • Have you updated hive-site.xml and verified the configuration gets picked up?
  • You should get a few jars from this build, hive-exec, hive-metastore, and the was-glue client jar.
  • Inspect your jar file to see if SessionHiveMetaStoreClientFactory is present in your hive-exec jar

Also as a disclaimer I'm just a contributor, not affiliated with AWS =)

@gaurang101197
Copy link

gaurang101197 commented May 30, 2020

@mitochon I figured out the problem by carefully looking at the patch. In this patch, there is not property called hive.metastore.client.factory.class but instead hive.imetastoreclient.factory.class property is used to provide client class name. I used hive.imetastoreclient.factory.class property in my hive-site.xml instead of hive.metastore.client.factory.class which is provided in README file. Then I got some class not found and method not found exception which I solved by downloading required aws-sdk jar and put them in auxlib folder of hive. And it is working fine now. Thanks. I think we should update property name in README so other don't fall in the same issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants