Mostly application developers bump into this issue. They ship their custom jars to map reduce job but when the classes in those are referred by code it throws a Class not found exception.
For -libjars to work your main class should satisfy the following two conditions.
1) Main Class should implement the Tool interface
//wrong usage - Tool Interface not implemented
//right usage
2) Main Class should get the existing configuration using getConf() method rather than creating anew configuration instance.
//wrong usage - creating anew instance of Conf
For -libjars to work your main class should satisfy the following two conditions.
1) Main Class should implement the Tool interface
//wrong usage - Tool Interface not implemented
public class WordCount extends Configured {
//right usage
public class WordCount extends Configured implements Tool {
2) Main Class should get the existing configuration using getConf() method rather than creating anew configuration instance.
//wrong usage - creating anew instance of Conf
public int run(String[] args) throws Exception {
Configuration conf = new Configuration();
//right usage public int run(String[] args) throws Exception {
Configuration conf = getConf();
My project is not working. I followed the steps given above, but i still get
ReplyDeleteException in thread "main" java.lang.ClassNotFoundException:
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:249)
at org.apache.hadoop.util.RunJar.main(RunJar.java:201).
Seems even the mainclass has not been found yet.
i use the following command:
hadoop jar ../lib/train_javaexample.jar train.examples.app.TypeSum -libjars=${LIBJAR} /user/whoiam/in /user/whoiam/out
proj code:
public class TypeSum extends Configured implements Tool{
...
public final int run(final String[] args) throws Exception {
Job job = new Job(super.getConf());
job.setJarByClass(TypeSum.class);
job.setMapperClass(TypeSumMapper.class);
job.setCombinerClass(TypeSumReducer.class);
job.setReducerClass(TypeSumReducer.class);
job.setOutputKeyClass(Text.class);
job.setOutputValueClass(IntWritable.class);
//FileInputFormat.addInputPath(job, new Path(args[0]));//???????????
//FileOutputFormat.setOutputPath(job, new Path(args[1]));
return (job.waitForCompletion(true)?0:1);
}
}
Could you help please?
nice post thanks for sharing Valuable information
ReplyDeleteThis comment has been removed by the author.
ReplyDeleteHadoop is really a good booming technology in coming days. And good path for the people who are looking for the Good Hikes and long career. We also provide Hadoop online training
ReplyDeleteExcellent Post thanks for sharing Hadoop Training in Hyderabad
ReplyDelete