[Lustre-discuss] Need help on lustre..

linux freaker linuxfreaker at gmail.com
Sun Feb 17 03:42:55 PST 2013


I tried running the below command but got the below error.
I have not put it into HDFS since Lustre is what I am trying to implement with.

[code]
#bin/hadoop jar hadoop-examples-1.1.1.jar wordcount
/user/hadoop/hadoop /user/hadoop-output

13/02/17 17:02:50 INFO util.NativeCodeLoader: Loaded the native-hadoop library
13/02/17 17:02:50 INFO input.FileInputFormat: Total input paths to process : 1
13/02/17 17:02:50 WARN snappy.LoadSnappy: Snappy native library not loaded
13/02/17 17:02:50 INFO mapred.JobClient: Cleaning up the staging area
file:/tmp/

hadoop-hadoop/mapred/staging/root/.staging/job_201302161113_0004
13/02/17 17:02:50 ERROR security.UserGroupInformation:
PriviledgedActionExceptio
                                          n as:root
cause:org.apache.hadoop.ipc.RemoteException: java.io.IOException: java

                .io.FileNotFoundException: File
file:/tmp/hadoop-hadoop/mapred/staging/root/.sta

ging/job_201302161113_0004/job.xml does not exist.
        at org.apache.hadoop.mapred.JobTracker.submitJob(JobTracker.java:3731)
        at org.apache.hadoop.mapred.JobTracker.submitJob(JobTracker.java:3695)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.

                java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces

                sorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:616)
        at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:578)
        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1393)
        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1389)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:416)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInforma

                tion.java:1136)
        at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1387)
Caused by: java.io.FileNotFoundException: File
file:/tmp/hadoop-hadoop/mapred/st

aging/root/.staging/job_201302161113_0004/job.xml does not exist.
        at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSys

                tem.java:397)
        at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.

                java:251)
        at org.apache.hadoop.mapred.JobInProgress.<init>(JobInProgress.java:406)
        at org.apache.hadoop.mapred.JobTracker.submitJob(JobTracker.java:3729)
        ... 12 more

org.apache.hadoop.ipc.RemoteException: java.io.IOException:
java.io.FileNotFound
                                     Exception: File
file:/tmp/hadoop-hadoop/mapred/staging/root/.staging/job_2013021

          61113_0004/job.xml does not exist.
        at org.apache.hadoop.mapred.JobTracker.submitJob(JobTracker.java:3731)
        at org.apache.hadoop.mapred.JobTracker.submitJob(JobTracker.java:3695)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.

                java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces

                sorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:616)
        at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:578)
        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1393)
        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1389)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:416)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInforma

                tion.java:1136)
        at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1387)
Caused by: java.io.FileNotFoundException: File
file:/tmp/hadoop-hadoop/mapred/st

aging/root/.staging/job_201302161113_0004/job.xml does not exist.
        at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSys

                tem.java:397)
        at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.

                java:251)
        at org.apache.hadoop.mapred.JobInProgress.<init>(JobInProgress.java:406)
        at org.apache.hadoop.mapred.JobTracker.submitJob(JobTracker.java:3729)
        ... 12 more

        at org.apache.hadoop.ipc.Client.call(Client.java:1107)
        at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:229)
        at org.apache.hadoop.mapred.$Proxy1.submitJob(Unknown Source)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.

                java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces

                sorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:616)
        at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryI

                nvocationHandler.java:85)
        at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocat

                ionHandler.java:62)
        at org.apache.hadoop.mapred.$Proxy1.submitJob(Unknown Source)
        at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:983)
        at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:912)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:416)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInforma

                tion.java:1136)
        at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:9

                12)
        at org.apache.hadoop.mapreduce.Job.submit(Job.java:500)
        at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:530)
        at org.apache.hadoop.examples.WordCount.main(WordCount.java:67)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.

                java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces

                sorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:616)
        at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(Progra

                mDriver.java:68)
        at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
        at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:64)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.

                java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces

                sorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:616)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
[root at alpha hadoop]#

[/code]

On 2/17/13, linux freaker <linuxfreaker at gmail.com> wrote:
> Hello,
>
> I have 4 machines - 1 MDS, 1 OSS, 2 Linux client. I need to run Hadoop
> over lustre replacing HDFS. All I have put the setup detail under
> http://paste.ubuntu.com/1661235/
>
> All I need to know is what I really need for Hadoop, what
> configuration changes are needed?
>  Please suggest.
>



More information about the lustre-discuss mailing list