Problem background
In the data platform, project construction requires the use of es and HBASE to build a data query interface. During the integration process, the jar package conflicts occur: com.google.common.base.Stopwatch.()V from class org.apache.hadoop.hbase.zookeeper.MetaTableLocator
org.apache.hadoop.hbase.DoNotRetryIOException: java.lang.IllegalAccessError: tried to access method com.google.common.base.Stopwatch.<init>()V from class org.apache.hadoop.hbase.zookeeper.MetaTableLocator at org.apache.hadoop.hbase.client.RpcRetryingCaller.translateException(RpcRetryingCaller.java:239) at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:150) ... at java.lang.Thread.run(Thread.java:745)Caused by: java.lang.IllegalAccessError: tried to access method com.google.common.base.Stopwatch.<init>()V from class org.apache.hadoop.hbase.zookeeper.MetaTableLocator at org.apache.hadoop.hbase.zookeeper.MetaTableLocator.blockUntil... at org.apache.hadoop.hbase.client.RegionServerCallable.prepare(RegionServerCallable.java:75) at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:134) ... 45 more
Solution
After investigation, it was confirmed that it was a conflict caused by the com.google.guava package. es depends on version 18 and above, while HBASE only supports version 16 and above. The guava package changes internally from 17 and the method changes, so 18 will not be backward compatible with version 16. During the project operation, if the 16 and 18 versions are introduced at the same time, the calling process of es and hbase will be confused. Then it will be easy to do next. We can repackage, type guava18 into es, and then display the package that references guava16 version in the pom file. In this way, es calls guava18 that is entered inside the package, while hbase calls guava16 that is entered outside.
Repackage
Create a new maven project and configure it in the pom file as follows:
<?xml version="1.0" encoding="UTF-8"?><project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> <modelVersion>4.0.0</modelVersion> <!-- this maven project is aim to fix the jar comflic in hbase & elasticsearch. in es, required guava 18+ or up. but in hbase you should use guava 16- or blow. cd the project directory and run 'sh ./cleanbuild.sh',you will get a new-self jar. then, adjust your project pom.xml whatth: <dependency> <groupId>douguo.shaded.elasticsearch</groupId> <artifactId>douguo_shaded_elasticsearch</artifactId> <version>1.0-SNAPSHOT</version> <exclusions> <exclusion> <groupId>org.elasticsearch</groupId> <artifactId>elasticsearch</artifactId> </exclusion> </exclusions> </dependency> finally, your jar comflic will be fixed. @date:2017-11-30 @author:zhangjianfei @since:douguo.shaded.elasticsearch-1.0.0 --> <groupId>douguo.shaded.elasticsearch</groupId> <artifactId>douguo_shaded_elasticsearch</artifactId> <version>1.0-SNAPSHOT</version> <properties> <elasticsearch.version>2.4.1</elasticsearch.version> </properties> <dependencies> <dependency> <groupId>org.elasticsearch</groupId> <artifactId>elasticsearch</artifactId> <version>${elasticsearch.version}</version> </dependency> <dependency> <groupId>org.elasticsearch.plugin</groupId> <artifactId>shield</artifactId> <version>${elasticsearch.version}</version> </dependency> </dependencies> <build> <plugins> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-shade-plugin</artifactId> <version>2.4.1</version> <configuration> <createDependencyReducedPom>false</createDependencyReducedPom> </configuration> <execution> <phase>package</phase> <goals> <goal>shade</goal> </goals> <configuration> <relocations> <relocation> <pattern>com.google.guava</pattern> <shadedPattern>douguo.shaded.elasticsearch.guava</shadedPattern> </relocation> <relocation> <pattern>org.joda</pattern> <shadedPattern>douguo.shaded.elasticsearch.joda</shadedPattern> </relocation> <relocation> <pattern>com.google.common</pattern> <shadedPattern>douguo.shaded.elasticsearch.common</shadedPattern> </relocation> <relocation> <pattern>com.google.thirdparty</pattern> <shadedPattern>douguo.shaded.elasticsearch.thirdparty</shadedPattern> </relocation> </relocations> <transformers> <transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer" /> </transformers> </configuration> </execution> </executions> </plugin> </build> <repositories> <repository> <id>elasticsearch-releases</id> <url>http://maven.elasticsearch.org/releases</url> <releases> <enabled>true</enabled> <updatePolicy>daily</updatePolicy> </releases> <snapshots> <enabled>false</enabled> </snapshots> </repository> </repository> </repositories></project> Migrate org.joda and other 4 possible conflicting jar packages through the maven-shade-plugin plugin and re-type jar packages so that when introducing this jar package, you can use the jar package's own dependencies instead of using external dependencies. It should be noted here that all com.google.common and other 4 packages need to be remigrated, otherwise java.lang.IllegalAccessError: tried to access method com.google.common.base error will occur.
Project package
mvn clean install
The new dependency package will be in the .m2 maven warehouse. If the company builds the warehouse, it is necessary to upload the jar package. If you run the jar package directly, remember to recompile the project and replace the lib directory
Project loads new package
Just configure it in the pom file:
<!-- douguo.shaded.elasticsearch --><dependency> <groupId>douguo.shaded.elasticsearch</groupId> <artifactId>douguo_shaded_elasticsearch</artifactId> <version>1.0-SNAPSHOT</version> <exclusions> <exclusion> <groupId>org.elasticsearch</groupId> <artifactId>elasticsearch</artifactId> </exclusion> </exclusions></dependency><!-- https://mvnrepository.com/artifact/com.google.guava/guava --><!-- this guava is only used in habse in es, 18.0+ is required, but hbase only supported 16.0 or blow. clean as install douguo.shaded.elasticsearch --><dependency> <groupId>com.google.guava</groupId> <artifactId>guava</artifactId> <version>16.0</version></dependency>
In this way, the guava18 package is under douguo.shaded.elasticsearch, and es will be called first. The externally configured guava16 will be called by HBASE. The 2 versions of jar packages exist independently of each other!
At this point, the problem is solved!
The above is all the content of this article. I hope it will be helpful to everyone's learning and I hope everyone will support Wulin.com more.