What's the best practice to build java-test-image? #16633

tisonkun · 2022-07-17T11:27:01Z

tisonkun
Jul 17, 2022
Collaborator

I ever asked this question on Slack but since I still met some issues I'd prefer to continue the discussion here so that we may get more attentions and the result would be publicly searchable.

@lhotari suggested to build the image with one line command build/build_java_test_image.sh on the thread.

After I cleaned up all states and try the command, it failed on:

[INFO] loaded credentials for service account with clientEmail=gharchive@white-ground-340901.iam.gserviceaccount.com
[INFO] dockerfile: null
[INFO] contextDirectory: /Users/tison/Brittani/pulsar/tests/docker-images/java-test-image
[INFO] Building Docker context /Users/tison/Brittani/pulsar/tests/docker-images/java-test-image
[INFO] Path(dockerfile): null
[INFO] Path(contextDirectory): /Users/tison/Brittani/pulsar/tests/docker-images/java-test-image
[INFO] 
[INFO] Image will be built as apachepulsar/java-test-image:2.11.0-SNAPSHOT
[INFO] 
[INFO] Step 1/25 : FROM ubuntu:20.04
[INFO] 
[INFO]  ---> 20fffa419e3a
[INFO] Step 2/25 : RUN groupadd -g 10001 pulsar
[INFO] 
[INFO]  ---> Running in eafa3773c178
[INFO] Removing intermediate container eafa3773c178
[INFO]  ---> 49de994567eb
[INFO] Step 3/25 : RUN adduser -u 10000 --gid 10001 --disabled-login --disabled-password --gecos '' pulsar
[INFO] 
[INFO]  ---> Running in 2ae29f463c4b
[INFO] Adding user `pulsar' ...
[INFO] Adding new user `pulsar' (10000) with group `pulsar' ...
[INFO] Creating home directory `/home/pulsar' ...
[INFO] Copying files from `/etc/skel' ...
[INFO] Removing intermediate container 2ae29f463c4b
[INFO]  ---> f04165752297
[INFO] Step 4/25 : ARG PULSAR_TARBALL=target/pulsar-server-distribution-bin.tar.gz
[INFO] 
[INFO]  ---> Running in 4292ea941c7b
[INFO] Removing intermediate container 4292ea941c7b
[INFO]  ---> 9e2a8c6baca6
[INFO] Step 5/25 : ADD ${PULSAR_TARBALL} /
[INFO] 
[INFO]  ---> 17cca323350f
[INFO] Step 6/25 : RUN mv /apache-pulsar-* /pulsar
[INFO] 
[INFO]  ---> Running in f69314c82fb8
[INFO] Removing intermediate container f69314c82fb8
[INFO]  ---> 15a2d2c3c202
[INFO] Step 7/25 : RUN chown -R root:root /pulsar
[INFO] 
[INFO]  ---> Running in 169a69ad93ee
[INFO] Removing intermediate container 169a69ad93ee
[INFO]  ---> 350c9ddab84c
[INFO] Step 8/25 : COPY target/scripts /pulsar/bin
[INFO] 
[INFO]  ---> 0d65cc27113c
[INFO] Step 9/25 : RUN chmod a+rx /pulsar/bin/*
[INFO] 
[INFO]  ---> Running in 2ca58ba18d58
[INFO] Removing intermediate container 2ca58ba18d58
[INFO]  ---> ced5c329b6f7
[INFO] Step 10/25 : WORKDIR /pulsar
[INFO] 
[INFO]  ---> Running in eb37b022e4a8
[INFO] Removing intermediate container eb37b022e4a8
[INFO]  ---> 7d9845313326
[INFO] Step 11/25 : ARG DEBIAN_FRONTEND=noninteractive
[INFO] 
[INFO]  ---> Running in ed8ae48c5248
[INFO] Removing intermediate container ed8ae48c5248
[INFO]  ---> de3616ffa7ac
[INFO] Step 12/25 : ARG UBUNTU_MIRROR=mirror://mirrors.ubuntu.com/mirrors.txt
[INFO] 
[INFO]  ---> Running in 4652059368fb
[INFO] Removing intermediate container 4652059368fb
[INFO]  ---> 80053cb922f9
[INFO] Step 13/25 : RUN sed -i "s|http://archive\.ubuntu\.com/ubuntu/|${UBUNTU_MIRROR:-mirror://mirrors.ubuntu.com/mirrors.txt}|g" /etc/apt/sources.list      && apt-get update      && apt-get -y dist-upgrade      && apt-get -y install openjdk-17-jdk-headless
[INFO] 
[INFO]  ---> Running in 1caab33a3d8a
[INFO] Get:1 http://security.ubuntu.com/ubuntu focal-security InRelease [114 kB]
[INFO] Get:2 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [883 kB]
[INFO] Get:3 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [27.5 kB]
[INFO] Get:4 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [1398 kB]
[INFO] Get:5 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [2027 kB]
[INFO] Err:6 http://mirrors.ubuntu.com/mirrors.txt Mirrorlist
[INFO]   Could not connect to mirrors.ubuntu.com:80 (198.19.194.95), connection timed out
[INFO] Ign:7 mirror://mirrors.ubuntu.com/mirrors.txt focal InRelease
[INFO] Ign:8 mirror://mirrors.ubuntu.com/mirrors.txt focal-updates InRelease
[INFO] Ign:9 mirror://mirrors.ubuntu.com/mirrors.txt focal-backports InRelease
[INFO] Err:10 mirror://mirrors.ubuntu.com/mirrors.txt focal Release
[INFO]   Downloading mirror file failed
[INFO] Err:11 mirror://mirrors.ubuntu.com/mirrors.txt focal-updates Release
[INFO]   Downloading mirror file failed
[INFO] Err:12 mirror://mirrors.ubuntu.com/mirrors.txt focal-backports Release
[INFO]   Downloading mirror file failed
[INFO] Reading package lists...
[INFO] 
[INFO] E: The repository 'mirror://mirrors.ubuntu.com/mirrors.txt focal Release' does not have a Release file.
[INFO] E: The repository 'mirror://mirrors.ubuntu.com/mirrors.txt focal-updates Release' does not have a Release file.
[INFO] E: The repository 'mirror://mirrors.ubuntu.com/mirrors.txt focal-backports Release' does not have a Release file.
[INFO] 
[ERROR] The command '/bin/sh -c sed -i "s|http://archive\.ubuntu\.com/ubuntu/|${UBUNTU_MIRROR:-mirror://mirrors.ubuntu.com/mirrors.txt}|g" /etc/apt/sources.list      && apt-get update      && apt-get -y dist-upgrade      && apt-get -y install openjdk-17-jdk-headless' returned a non-zero code: 100
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for Pulsar 2.11.0-SNAPSHOT:
[INFO] 
[INFO] Pulsar Build Tools ................................. SUCCESS [  2.783 s]
[INFO] Pulsar ............................................. SUCCESS [  1.016 s]
[INFO] Pulsar Client :: API ............................... SUCCESS [  1.044 s]
[INFO] Pulsar Client Admin :: API ......................... SUCCESS [  0.499 s]
[INFO] Pulsar Common ...................................... SUCCESS [  8.228 s]
[INFO] Apache Pulsar :: Bouncy Castle :: Parent ........... SUCCESS [  0.153 s]
[INFO] Apache Pulsar :: Bouncy Castle :: BC ............... SUCCESS [  0.413 s]
[INFO] pulsar-client-messagecrypto-bc ..................... SUCCESS [  0.187 s]
[INFO] Pulsar Functions :: Parent ......................... SUCCESS [  0.147 s]
[INFO] Pulsar Functions :: Proto .......................... SUCCESS [  0.358 s]
[INFO] Pulsar Client Java ................................. SUCCESS [  1.152 s]
[INFO] Pulsar Metadata .................................... SUCCESS [  0.700 s]
[INFO] pulsar-broker-common ............................... SUCCESS [  0.767 s]
[INFO] Pulsar Test Mocks .................................. SUCCESS [  0.460 s]
[INFO] Managed Ledger ..................................... SUCCESS [  0.568 s]
[INFO] Pulsar WebSocket ................................... SUCCESS [  0.800 s]
[INFO] Apache Pulsar :: Package Management ................ SUCCESS [  0.170 s]
[INFO] Apache Pulsar :: Package Management :: Core ........ SUCCESS [  0.179 s]
[INFO] Pulsar Client Admin Original ....................... SUCCESS [  0.555 s]
[INFO] Pulsar Transaction :: Parent ....................... SUCCESS [  0.199 s]
[INFO] Pulsar Transaction :: Common ....................... SUCCESS [  0.370 s]
[INFO] Pulsar Functions :: API ............................ SUCCESS [  0.225 s]
[INFO] Pulsar IO :: Parent ................................ SUCCESS [  0.187 s]
[INFO] Pulsar IO :: IO .................................... SUCCESS [  0.223 s]
[INFO] Pulsar IO :: Batch Discovery Triggerers ............ SUCCESS [  0.260 s]
[INFO] pulsar-config-validation ........................... SUCCESS [  0.183 s]
[INFO] Pulsar Functions :: Utils .......................... SUCCESS [  0.387 s]
[INFO] Pulsar IO :: IO Common ............................. SUCCESS [  0.276 s]
[INFO] Pulsar Functions :: Secrets ........................ SUCCESS [  0.346 s]
[INFO] Pulsar Functions :: Instance ....................... SUCCESS [  1.112 s]
[INFO] Pulsar Functions :: Runtime ........................ SUCCESS [  0.884 s]
[INFO] Pulsar IO :: Cassandra ............................. SUCCESS [  1.248 s]
[INFO] Pulsar IO :: Twitter ............................... SUCCESS [  1.062 s]
[INFO] Pulsar Functions :: API Examples ................... SUCCESS [  0.232 s]
[INFO] Pulsar Functions :: API Examples (NAR) ............. SUCCESS [  0.173 s]
[INFO] Pulsar Functions :: Worker ......................... SUCCESS [  1.556 s]
[INFO] Pulsar Functions :: Local Runner original .......... SUCCESS [  0.884 s]
[INFO] Pulsar IO :: Batch Data Generator .................. SUCCESS [  1.312 s]
[INFO] Pulsar IO :: Data Generator ........................ SUCCESS [  0.323 s]
[INFO] Pulsar Transaction :: Coordinator .................. SUCCESS [  0.926 s]
[INFO] Apache Pulsar :: Package Management :: Filesystem Storage SUCCESS [  0.282 s]
[INFO] Pulsar Broker ...................................... SUCCESS [  2.443 s]
[INFO] Pulsar Proxy ....................................... SUCCESS [  1.052 s]
[INFO] pulsar-client-auth-sasl ............................ SUCCESS [  0.303 s]
[INFO] pulsar-broker-auth-sasl ............................ SUCCESS [  1.236 s]
[INFO] Apache Pulsar :: Package Management :: BookKeeper Storage SUCCESS [  0.703 s]
[INFO] Pulsar Client Tools ................................ SUCCESS [  0.945 s]
[INFO] Pulsar Test Client ................................. SUCCESS [  1.342 s]
[INFO] Pulsar Functions :: Runtime All .................... SUCCESS [  2.771 s]
[INFO] Pulsar :: Distribution ............................. SUCCESS [  0.127 s]
[INFO] Pulsar :: Distribution :: Server ................... SUCCESS [ 11.002 s]
[INFO] Apache Pulsar :: Tests ............................. SUCCESS [  0.138 s]
[INFO] Apache Pulsar :: Tests :: Docker Images ............ SUCCESS [  0.114 s]
[INFO] Apache Pulsar :: Tests :: Docker Images :: Java Test Functions SUCCESS [  0.331 s]
[INFO] Apache Pulsar :: Tests :: Docker Images :: Java Test Image FAILURE [13:26 min]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  14:22 min
[INFO] Finished at: 2022-07-17T19:19:25+08:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal com.spotify:dockerfile-maven-plugin:1.4.13:build (default) on project java-test-image: Could not build image: The command '/bin/sh -c sed -i "s|http://archive\.ubuntu\.com/ubuntu/|${UBUNTU_MIRROR:-mirror://mirrors.ubuntu.com/mirrors.txt}|g" /etc/apt/sources.list      && apt-get update      && apt-get -y dist-upgrade      && apt-get -y install openjdk-17-jdk-headless' returned a non-zero code: 100 -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <args> -rf :java-test-image

The entire output are too many to paste here. If you need full log, let me know and I can attach one via gist.

I don't know whether this failure is a network issue or some settings I should correct.

Answered by tisonkun

Jul 18, 2022

Each time it failed in different step but all about cannot download from a remote place. Perhaps it's about network issues.

View full answer

tisonkun · 2022-07-18T04:51:42Z

tisonkun
Jul 18, 2022
Collaborator Author

Each time it failed in different step but all about cannot download from a remote place. Perhaps it's about network issues.

4 replies

lhotari Jul 18, 2022
Collaborator

There seems to be some network issues. In the output, you can see:

[INFO] Err:6 http://mirrors.ubuntu.com/mirrors.txt Mirrorlist
[INFO] Could not connect to mirrors.ubuntu.com:80 (198.19.194.95), connection timed out

By default, the docker image uses http://mirrors.ubuntu.com/mirrors.txt as the source for Ubuntu mirrors.
You can manually choose the mirror by setting UBUNTU_MIRROR environment variable. You can find the listing of mirrors by country in http://mirrors.ubuntu.com . For example, http://mirrors.ubuntu.com/US.txt, http://mirrors.ubuntu.com/CN.txt

# example of using the Ubuntu default mirror
UBUNTU_MIRROR=http://archive.ubuntu.com/ubuntu/ ./build/build_java_test_image.sh

# example of using a specific mirror from http://mirrors.ubuntu.com/CN.txt
UBUNTU_MIRROR=http://mirrors.huaweicloud.com/repository/ubuntu/ ./build/build_java_test_image.sh

I hope this helps.

tisonkun Jul 18, 2022
Collaborator Author

Thank you @lhotari! I'll try this script later.

tisonkun Jul 18, 2022
Collaborator Author

The first script works! Thank you.

tisonkun Jul 18, 2022
Collaborator Author

@lhotari the next time you can answer in a different thread so that I can mark your response as the answer :)

Navigation Menu

Uh oh!

What's the best practice to build java-test-image? #16633

Uh oh!

{{title}}

Uh oh!

Replies: 1 comment 4 replies

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Select a reply

Uh oh!

Uh oh!

What's the best practice to build java-test-image? #16633

Uh oh!

tisonkun Jul 17, 2022 Collaborator

Replies: 1 comment · 4 replies

Uh oh!

tisonkun Jul 18, 2022 Collaborator Author

Uh oh!

lhotari Jul 18, 2022 Collaborator

Uh oh!

tisonkun Jul 18, 2022 Collaborator Author

Uh oh!

tisonkun Jul 18, 2022 Collaborator Author

Uh oh!

tisonkun Jul 18, 2022 Collaborator Author

tisonkun
Jul 17, 2022
Collaborator

Replies: 1 comment 4 replies

tisonkun
Jul 18, 2022
Collaborator Author

lhotari Jul 18, 2022
Collaborator

tisonkun Jul 18, 2022
Collaborator Author

tisonkun Jul 18, 2022
Collaborator Author

tisonkun Jul 18, 2022
Collaborator Author