Debian host cannot reach bionic/universe in make-base-vm

[willyko]

Host: Debian 9
Hitting the following error when creating bionic lxc vm. Possibly an issue with communicating with apt-cacher-ng?

$ ./bin/make-base-vm --suite bionic --arch amd64 --lxc
I: Retrieving InRelease 
I: Checking Release signature
I: Valid Release signature (key id 790BC7277767219C42C86F933B4FE6ACC0B21F32)
I: Retrieving Packages 
I: Retrieving Packages 
I: Validating Packages 
I: Retrieving Packages 
I: Retrieving Packages 
I: Retrieving Packages 
E: Couldn't download dists/bionic/universe/binary-amd64/Packages

I could bypass this this error by removing the $MIRROR at the end of

gitian-builder/bin/make-base-vm

Line 261 in c0f77ca

env -i LANG=en_US.UTF-8 LC_ALL=en_US.UTF-8 DEBOOTSTRAP_DIR="$DEBOOTSTRAP_DIR" sudo ${preserve_env+--preserve-env} debootstrap --arch=$ARCH --include=$addpkg --exclude=$removepkg --components=$components $SUITE $OUT-bootstrap $MIRROR

[Sharpiro]

I also had a similar issue when running podman (docker) as the host. Removing the $MIRROR also fixed that. I'm not even sure what a valid MIRROR_HOST env variable would look like or how to point it at a default Ubuntu server in order to override this. I was getting an error saying 127.0.0.1/... is unreachable, so I think for whatever reason the script defaults to localhost, when running inside a docker/podman container, this is not applicable.

[decryp2kanon]

same here on debian 9.5 and 10.5

$ bin/make-base-vm --lxc --arch amd64 --suite bionic

I: Retrieving InRelease 
I: Checking Release signature
I: Valid Release signature (key id 790BC7277767219C42C86F933B4FE6ACC0B21F32)
I: Retrieving Packages 
I: Retrieving Packages 
I: Retrieving Packages 
E: Couldn't download dists/bionic/main/binary-amd64/Packages

FYI, its working on ubuntu 18.04 with no any modification.

[devrandom]

it's expecting apt-cacher-ng to be listening on localhost. if you run make-base-vm without arguments, there's help on this topic.

[decryp2kanon]

if you run make-base-vm without arguments, there's help on this topic.

do you mean, run bin/make-base-vm instead of bin/make-base-vm --lxc --arch amd64 --suite bionic? @devrandom

$ bin/make-base-vm
2020-09-23 11:39:08,377 INFO    : logging to file: /tmp/tmpd_GySy
2020-09-23 11:39:08,445 INFO    : Calling hook: preflight_check
2020-09-23 11:39:08,459 INFO    : Calling hook: set_defaults
2020-09-23 11:39:08,460 INFO    : Calling hook: bootstrap
2020-09-23 11:39:09,977 INFO    : Cleaning up
2020-09-23 11:39:09,977 ERROR   : Process (['/usr/sbin/debootstrap', '--arch=amd64', 'xenial', '/tmp/tmpM7b9mN', 'http://127.0.0.1:3142/archive.ubuntu.com/ubuntu']) returned 1. stdout: I: Retrieving InRelease 
I: Checking Release signature
I: Valid Release signature (key id 790BC7277767219C42C86F933B4FE6ACC0B21F32)
I: Retrieving Packages 
I: Retrieving Packages 
I: Retrieving Packages 
E: Couldn't download dists/xenial/main/binary-amd64/Packages
, stderr: 
Traceback (most recent call last):
  File "/usr/local/bin/vmbuilder", line 24, in <module>
    cli.main()
  File "/usr/local/lib/python2.7/dist-packages/VMBuilder/contrib/cli.py", line 221, in main
    distro.build_chroot()
  File "/usr/local/lib/python2.7/dist-packages/VMBuilder/distro.py", line 83, in build_chroot
    self.call_hooks('bootstrap')
  File "/usr/local/lib/python2.7/dist-packages/VMBuilder/distro.py", line 67, in call_hooks
    call_hooks(self, *args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/VMBuilder/util.py", line 170, in call_hooks
    getattr(context, func)(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/VMBuilder/plugins/ubuntu/distro.py", line 139, in bootstrap
    self.suite.debootstrap()
  File "/usr/local/lib/python2.7/dist-packages/VMBuilder/plugins/ubuntu/dapper.py", line 273, in debootstrap
    run_cmd(*cmd, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/VMBuilder/util.py", line 120, in run_cmd
    raise VMBuilderException, "Process (%s) returned %d. stdout: %s, stderr: %s" % (args.__repr__(), status, mystdout.buf, mystderr.buf)
VMBuilder.exception.VMBuilderException: Process (['/usr/sbin/debootstrap', '--arch=amd64', 'xenial', '/tmp/tmpM7b9mN', 'http://127.0.0.1:3142/archive.ubuntu.com/ubuntu']) returned 1. stdout: I: Retrieving InRelease 
I: Checking Release signature
I: Valid Release signature (key id 790BC7277767219C42C86F933B4FE6ACC0B21F32)
I: Retrieving Packages 
I: Retrieving Packages 
I: Retrieving Packages 
E: Couldn't download dists/xenial/main/binary-amd64/Packages
, stderr:

[decryp2kanon]

if i remove $MIRROR on line 261, its started, but i got error soon.

debian 9.5

Updating apt-get repository (log in var/install.log)
bin/gbuild:23:in `system!': failed to run on-target -u root apt-get update >> var/install.log 2>&1 (RuntimeError)
	from bin/gbuild:117:in `build_one_configuration'
	from bin/gbuild:336:in `block (2 levels) in <main>'
	from bin/gbuild:331:in `each'
	from bin/gbuild:331:in `block in <main>'
	from bin/gbuild:329:in `each'
	from bin/gbuild:329:in `<main>'
Traceback (most recent call last):
  File "./gitian-build.py", line 262, in <module>
    main()
  File "./gitian-build.py", line 250, in main
    build()
  File "./gitian-build.py", line 63, in build
    subprocess.check_call(['bin/gbuild', '-j', args.jobs, '-m', args.memory, '--commit', 'bitcoin='+args.commit, '--url', 'bitcoin='+args.url, '../bitcoin/contrib/gitian-descriptors/gitian-linux.yml'])
  File "/usr/lib/python3.5/subprocess.py", line 271, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['bin/gbuild', '-j', '4', '-m', '8000', '--commit', 'bitcoin=v0.20.1', '--url', 'bitcoin=https://github.com/bitcoin/bitcoin', '../bitcoin/contrib/gitian-descriptors/gitian-linux.yml']' returned non-zero exit status 1

debian 10.5

Updating apt-get repository (log in var/install.log)
Traceback (most recent call last):
	6: from bin/gbuild:329:in `<main>'
	5: from bin/gbuild:329:in `each'
	4: from bin/gbuild:331:in `block in <main>'
	3: from bin/gbuild:331:in `each'
	2: from bin/gbuild:336:in `block (2 levels) in <main>'
	1: from bin/gbuild:117:in `build_one_configuration'
bin/gbuild:23:in `system!': failed to run on-target -u root apt-get update >> var/install.log 2>&1 (RuntimeError)
Traceback (most recent call last):
  File "./gitian-build.py", line 262, in <module>
    main()
  File "./gitian-build.py", line 250, in main
    build()
  File "./gitian-build.py", line 63, in build
    subprocess.check_call(['bin/gbuild', '-j', args.jobs, '-m', args.memory, '--commit', 'bitcoin='+args.commit, '--url', 'bitcoin='+args.url, '../bitcoin/contrib/gitian-descriptors/gitian-linux.yml'])
  File "/usr/lib/python3.7/subprocess.py", line 347, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['bin/gbuild', '-j', '4', '-m', '8000', '--commit', 'bitcoin=v0.20.1', '--url', 'bitcoin=https://github.com/bitcoin/bitcoin', '../bitcoin/contrib/gitian-descriptors/gitian-linux.yml']' returned non-zero exit status 1.

[devrandom]

do you mean, run bin/make-base-vm instead of bin/make-base-vm --lxc --arch amd64 --suite bionic?

Yes. You will see the help message.

[decryp2kanon]

FYI, I succeed on ubuntu 18.04.
https://gist.github.com/decryp2kanon/67730da9e5f78bfb07f555d106b36686