builder: release-mozilla-beta-fennec_bouncer_submitter slave: bld-linux64-spot-433 starttime: 1470771192.58 results: success (0) buildid: 20160809123302 builduid: 1350492489754fe0a315d9e8516c5e5b ========= Started set props: master (results: 0, elapsed: 0 secs) (at 2016-08-09 12:33:12.579138) ========= master: http://buildbot-master91.bb.releng.usw2.mozilla.com:8001/ ========= Finished set props: master (results: 0, elapsed: 0 secs) (at 2016-08-09 12:33:12.579556) ========= ========= Started downloading to buildprops.json (results: 0, elapsed: 0 secs) (at 2016-08-09 12:33:12.579835) ========= ========= Finished downloading to buildprops.json (results: 0, elapsed: 0 secs) (at 2016-08-09 12:33:12.593232) ========= ========= Started 'bash -c ...' (results: 0, elapsed: 0 secs) (at 2016-08-09 12:33:12.593505) ========= bash -c 'wget -Oarchiver_client.py --no-check-certificate --tries=10 --waitretry=3 https://hg.mozilla.org/build/tools/raw-file/default/buildfarm/utils/archiver_client.py' in dir /builds/slave/rel-m-beta-bncr_sub-0000000000/. (timeout 1200 secs) watching logfiles {} argv: ['bash', '-c', 'wget -Oarchiver_client.py --no-check-certificate --tries=10 --waitretry=3 https://hg.mozilla.org/build/tools/raw-file/default/buildfarm/utils/archiver_client.py'] environment: BUILDBOTCUSTOM=https://hg.mozilla.org/build/buildbotcustom BUILDBOT_CONFIGS=https://hg.mozilla.org/build/buildbot-configs CCACHE_HASHDIR= CLOBBERER_URL=https://api.pub.build.mozilla.org/clobberer/forceclobber G_BROKEN_FILENAMES=1 HISTCONTROL=ignoredups HISTSIZE=1000 HOME=/home/cltbld HOSTNAME=bld-linux64-spot-433.build.releng.usw2.mozilla.com LANG=en_US.UTF-8 LESSOPEN=|/usr/bin/lesspipe.sh %s LOGNAME=cltbld MAIL=/var/spool/mail/cltbld PATH=/usr/local/bin:/usr/lib64/ccache:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/cltbld/bin PWD=/builds/slave/rel-m-beta-bncr_sub-0000000000 SHELL=/bin/bash SHLVL=1 TERM=linux TMOUT=86400 USER=cltbld _=/tools/buildbot/bin/python using PTY: False --2016-08-09 12:33:12-- https://hg.mozilla.org/build/tools/raw-file/default/buildfarm/utils/archiver_client.py Resolving hg.mozilla.org... 63.245.215.102, 63.245.215.25 Connecting to hg.mozilla.org|63.245.215.102|:443... connected. HTTP request sent, awaiting response... 200 Script output follows Length: 12141 (12K) [text/x-python] Saving to: ‘archiver_client.py’ 0K .......... . 100% 103M=0s 2016-08-09 12:33:12 (103 MB/s) - ‘archiver_client.py’ saved [12141/12141] program finished with exit code 0 elapsedTime=0.201005 ========= master_lag: 0.01 ========= ========= Finished 'bash -c ...' (results: 0, elapsed: 0 secs) (at 2016-08-09 12:33:12.808290) ========= ========= Started 'rm -rf ...' (results: 0, elapsed: 0 secs) (at 2016-08-09 12:33:12.808571) ========= rm -rf scripts properties in dir /builds/slave/rel-m-beta-bncr_sub-0000000000/. (timeout 1200 secs) watching logfiles {} argv: ['rm', '-rf', 'scripts', 'properties'] environment: BUILDBOTCUSTOM=https://hg.mozilla.org/build/buildbotcustom BUILDBOT_CONFIGS=https://hg.mozilla.org/build/buildbot-configs CCACHE_HASHDIR= CLOBBERER_URL=https://api.pub.build.mozilla.org/clobberer/forceclobber G_BROKEN_FILENAMES=1 HISTCONTROL=ignoredups HISTSIZE=1000 HOME=/home/cltbld HOSTNAME=bld-linux64-spot-433.build.releng.usw2.mozilla.com LANG=en_US.UTF-8 LESSOPEN=|/usr/bin/lesspipe.sh %s LOGNAME=cltbld MAIL=/var/spool/mail/cltbld PATH=/usr/local/bin:/usr/lib64/ccache:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/cltbld/bin PWD=/builds/slave/rel-m-beta-bncr_sub-0000000000 SHELL=/bin/bash SHLVL=1 TERM=linux TMOUT=86400 USER=cltbld _=/tools/buildbot/bin/python using PTY: False program finished with exit code 0 elapsedTime=0.003463 ========= master_lag: 0.07 ========= ========= Finished 'rm -rf ...' (results: 0, elapsed: 0 secs) (at 2016-08-09 12:33:12.878431) ========= ========= Started 'bash -c ...' (results: 0, elapsed: 3 mins, 38 secs) (at 2016-08-09 12:33:12.878735) ========= bash -c 'python archiver_client.py mozharness --repo releases/mozilla-beta --tag FENNEC_49_0b2_RELEASE --destination scripts --debug' in dir /builds/slave/rel-m-beta-bncr_sub-0000000000/. (timeout 1200 secs) watching logfiles {} argv: ['bash', '-c', 'python archiver_client.py mozharness --repo releases/mozilla-beta --tag FENNEC_49_0b2_RELEASE --destination scripts --debug'] environment: BUILDBOTCUSTOM=https://hg.mozilla.org/build/buildbotcustom BUILDBOT_CONFIGS=https://hg.mozilla.org/build/buildbot-configs CCACHE_HASHDIR= CLOBBERER_URL=https://api.pub.build.mozilla.org/clobberer/forceclobber G_BROKEN_FILENAMES=1 HISTCONTROL=ignoredups HISTSIZE=1000 HOME=/home/cltbld HOSTNAME=bld-linux64-spot-433.build.releng.usw2.mozilla.com LANG=en_US.UTF-8 LESSOPEN=|/usr/bin/lesspipe.sh %s LOGNAME=cltbld MAIL=/var/spool/mail/cltbld PATH=/usr/local/bin:/usr/lib64/ccache:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/cltbld/bin PWD=/builds/slave/rel-m-beta-bncr_sub-0000000000 SHELL=/bin/bash SHLVL=1 TERM=linux TMOUT=86400 USER=cltbld _=/tools/buildbot/bin/python using PTY: False 2016-08-09 12:33:12,935 Setting DEBUG logging. 2016-08-09 12:33:12,935 "FENNEC_49_0b2_RELEASE" was passed as the tag. Querying remote repository for corresponding rev hash. 2016-08-09 12:33:13,559 revision being used: ef8bd81c9848 2016-08-09 12:33:13,559 attempt 1/10 2016-08-09 12:33:13,559 Getting archive location from https://api.pub.build.mozilla.org/archiver/hgmo/releases/mozilla-beta/ef8bd81c9848?&preferred_region=us-west-2&suffix=tar.gz&subdir=testing/mozharness 2016-08-09 12:33:13,818 attempt 1/10 2016-08-09 12:33:14,208 current task status: no status available at this point. state: PENDING 2016-08-09 12:33:14,208 sleeping for 10.00s (attempt 1/10) 2016-08-09 12:33:24,219 attempt 2/10 2016-08-09 12:33:24,629 current task status: no status available at this point. state: PENDING 2016-08-09 12:33:24,629 sleeping for 15.00s (attempt 2/10) 2016-08-09 12:33:39,645 attempt 3/10 2016-08-09 12:33:40,042 current task status: no status available at this point. state: PENDING 2016-08-09 12:33:40,042 sleeping for 23.50s (attempt 3/10) 2016-08-09 12:34:03,566 attempt 4/10 2016-08-09 12:34:03,971 current task status: no status available at this point. state: PENDING 2016-08-09 12:34:03,971 sleeping for 35.25s (attempt 4/10) 2016-08-09 12:34:39,257 attempt 5/10 2016-08-09 12:34:39,676 current task status: no status available at this point. state: PENDING 2016-08-09 12:34:39,676 sleeping for 52.88s (attempt 5/10) 2016-08-09 12:35:32,578 attempt 6/10 2016-08-09 12:35:32,999 current task status: Task has expired from pending for too long. Re-creating task. state: RETRY 2016-08-09 12:35:32,999 sleeping for 78.31s (attempt 6/10) 2016-08-09 12:36:51,390 attempt 7/10 2016-08-09 12:36:51,566 current task status: Task completed! Check 's3_urls' for upload locations. state: SUCCESS 2016-08-09 12:36:51,640 unpacking tar archive at: mozilla-beta-ef8bd81c9848/testing/mozharness/ program finished with exit code 0 elapsedTime=218.875758 ========= master_lag: 0.01 ========= ========= Finished 'bash -c ...' (results: 0, elapsed: 3 mins, 38 secs) (at 2016-08-09 12:36:51.766549) ========= ========= Started set props: script_repo_revision (results: 0, elapsed: 0 secs) (at 2016-08-09 12:36:51.766871) ========= script_repo_revision: FENNEC_49_0b2_RELEASE ========= Finished set props: script_repo_revision (results: 0, elapsed: 0 secs) (at 2016-08-09 12:36:51.767259) ========= ========= Started downloading to oauth.txt (results: 0, elapsed: 0 secs) (at 2016-08-09 12:36:51.767538) ========= ========= Finished downloading to oauth.txt (results: 0, elapsed: 0 secs) (at 2016-08-09 12:36:51.776992) ========= ========= Started 'scripts/scripts/bouncer_submitter.py -c ...' (results: 0, elapsed: 2 secs) (at 2016-08-09 12:36:51.777266) ========= scripts/scripts/bouncer_submitter.py -c releases/bouncer_fennec.py --revision FENNEC_49_0b2_RELEASE --repo releases/mozilla-beta --version 49.0b2 --credentials-file oauth.txt --bouncer-api-prefix https://bounceradmin.mozilla.com/api --build-number 1 in dir /builds/slave/rel-m-beta-bncr_sub-0000000000/. (timeout 1200 secs) watching logfiles {} argv: ['scripts/scripts/bouncer_submitter.py', '-c', 'releases/bouncer_fennec.py', '--revision', 'FENNEC_49_0b2_RELEASE', '--repo', 'releases/mozilla-beta', '--version', '49.0b2', '--credentials-file', 'oauth.txt', '--bouncer-api-prefix', 'https://bounceradmin.mozilla.com/api', '--build-number', '1'] environment: BUILDBOTCUSTOM=https://hg.mozilla.org/build/buildbotcustom BUILDBOT_CONFIGS=https://hg.mozilla.org/build/buildbot-configs CCACHE_HASHDIR= CLOBBERER_URL=https://api.pub.build.mozilla.org/clobberer/forceclobber G_BROKEN_FILENAMES=1 HISTCONTROL=ignoredups HISTSIZE=1000 HOME=/home/cltbld HOSTNAME=bld-linux64-spot-433.build.releng.usw2.mozilla.com LANG=en_US.UTF-8 LESSOPEN=|/usr/bin/lesspipe.sh %s LOGNAME=cltbld MAIL=/var/spool/mail/cltbld PATH=/usr/local/bin:/usr/lib64/ccache:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/cltbld/bin PROPERTIES_FILE=/builds/slave/rel-m-beta-bncr_sub-0000000000/buildprops.json PWD=/builds/slave/rel-m-beta-bncr_sub-0000000000 SHELL=/bin/bash SHLVL=1 TERM=linux TMOUT=86400 USER=cltbld _=/tools/buildbot/bin/python using PTY: False 12:36:51 INFO - MultiFileLogger online at 20160809 12:36:51 in /builds/slave/rel-m-beta-bncr_sub-0000000000 12:36:51 INFO - Using buildbot properties: 12:36:51 INFO - { 12:36:51 INFO - "project": "", 12:36:51 INFO - "product": "Fennec", 12:36:51 INFO - "build_number": 1, 12:36:51 INFO - "scheduler": "release-mozilla-beta-fennec_build", 12:36:51 INFO - "repository": "", 12:36:51 INFO - "buildername": "release-mozilla-beta-fennec_bouncer_submitter", 12:36:51 INFO - "buildid": "20160809123302", 12:36:51 INFO - "basedir": "/builds/slave/rel-m-beta-bncr_sub-0000000000", 12:36:51 INFO - "release_config": "mozilla/release-fennec-mozilla-beta.py", 12:36:51 INFO - "buildnumber": 13, 12:36:51 INFO - "platform": null, 12:36:51 INFO - "version": "49.0b2", 12:36:51 INFO - "master": "http://buildbot-master91.bb.releng.usw2.mozilla.com:8001/", 12:36:51 INFO - "slavebuilddir": "rel-m-beta-bncr_sub-0000000000", 12:36:51 INFO - "branch": "release-mozilla-beta", 12:36:51 INFO - "script_repo_revision": "FENNEC_49_0b2_RELEASE", 12:36:51 INFO - "products": "fennec", 12:36:51 INFO - "slavename": "bld-linux64-spot-433", 12:36:51 INFO - "builduid": "1350492489754fe0a315d9e8516c5e5b", 12:36:51 INFO - "revision": null 12:36:51 INFO - } 12:36:51 INFO - Run as scripts/scripts/bouncer_submitter.py -c releases/bouncer_fennec.py --revision FENNEC_49_0b2_RELEASE --repo releases/mozilla-beta --version 49.0b2 --credentials-file oauth.txt --bouncer-api-prefix https://bounceradmin.mozilla.com/api --build-number 1 12:36:51 INFO - Dumping config to /builds/slave/rel-m-beta-bncr_sub-0000000000/logs/localconfig.json. 12:36:51 INFO - {'append_to_log': False, 12:36:51 INFO - 'base_work_dir': '/builds/slave/rel-m-beta-bncr_sub-0000000000', 12:36:51 INFO - 'bouncer-api-prefix': 'https://bounceradmin.mozilla.com/api', 12:36:51 INFO - 'build_number': '1', 12:36:51 INFO - 'buildbot_json_path': 'buildprops.json', 12:36:51 INFO - 'config_files': ('releases/bouncer_fennec.py',), 12:36:51 INFO - 'credentials_file': 'oauth.txt', 12:36:51 INFO - 'log_level': 'info', 12:36:51 INFO - 'log_to_console': True, 12:36:51 INFO - 'opt_config_files': (), 12:36:51 INFO - 'products': {'apk': {'add-locales': False, 12:36:51 INFO - 'alias': 'fennec-latest', 12:36:51 INFO - 'check_uptake': True, 12:36:51 INFO - 'paths': {'android-api-15': {'bouncer-platform': 'android', 12:36:51 INFO - 'path': '/mobile/releases/%(version)s/android-api-15/:lang/fennec-%(version)s.:lang.android-arm.apk'}, 12:36:51 INFO - 'android-x86': {'bouncer-platform': 'android-x86', 12:36:51 INFO - 'path': '/mobile/releases/%(version)s/android-x86/:lang/fennec-%(version)s.:lang.android-i386.apk'}}, 12:36:51 INFO - 'product-name': 'Fennec-%(version)s', 12:36:51 INFO - 'ssl-only': False}}, 12:36:51 INFO - 'repo': 'releases/mozilla-beta', 12:36:51 INFO - 'revision': 'FENNEC_49_0b2_RELEASE', 12:36:51 INFO - 'version': '49.0b2', 12:36:51 INFO - 'volatile_config': {'actions': None, 'add_actions': None, 'no_actions': None}, 12:36:51 INFO - 'work_dir': 'build'} 12:36:51 INFO - ##### 12:36:51 INFO - ##### Running clobber step. 12:36:51 INFO - ##### 12:36:51 INFO - Running main action method: clobber 12:36:51 INFO - rmtree: /builds/slave/rel-m-beta-bncr_sub-0000000000/build 12:36:51 INFO - ##### 12:36:51 INFO - ##### Finished clobber step (success) 12:36:51 INFO - ##### 12:36:51 INFO - ##### 12:36:51 INFO - ##### Running download-shipped-locales step. 12:36:51 INFO - ##### 12:36:51 INFO - Running main action method: download_shipped_locales 12:36:51 INFO - No need to download shipped-locales 12:36:51 INFO - ##### 12:36:51 INFO - ##### Finished download-shipped-locales step (success) 12:36:51 INFO - ##### 12:36:51 INFO - ##### 12:36:51 INFO - ##### Running submit step. 12:36:51 INFO - ##### 12:36:51 INFO - Running main action method: submit 12:36:51 INFO - Checking if Fennec-49.0b2 already exists 12:36:51 INFO - retry: Calling _api_call with args: ('product_show?product=Fennec-49.0b2', None), kwargs: {}, attempt #1 12:36:51 INFO - Submitting to https://bounceradmin.mozilla.com/api/product_show?product=Fennec-49.0b2 12:36:52 INFO - Server response 12:36:52 INFO - 12:36:52 INFO - Products found: 0 12:36:52 INFO - Adding apk... 12:36:52 INFO - retry: Calling _api_call with args: ('product_add/', {'product': 'Fennec-49.0b2'}), kwargs: {}, attempt #1 12:36:52 INFO - POST data: product=Fennec-49.0b2 12:36:52 INFO - Submitting to https://bounceradmin.mozilla.com/api/product_add/ 12:36:53 INFO - Server response 12:36:53 INFO - 12:36:53 INFO - Adding paths... 12:36:53 INFO - android-api-15 (android): /mobile/releases/49.0b2/android-api-15/:lang/fennec-49.0b2.:lang.android-arm.apk 12:36:53 INFO - retry: Calling _api_call with args: ('location_add/', {'path': '/mobile/releases/49.0b2/android-api-15/:lang/fennec-49.0b2.:lang.android-arm.apk', 'product': 'Fennec-49.0b2', 'os': 'android'}), kwargs: {}, attempt #1 12:36:53 INFO - POST data: path=%2Fmobile%2Freleases%2F49.0b2%2Fandroid-api-15%2F%3Alang%2Ffennec-49.0b2.%3Alang.android-arm.apk&product=Fennec-49.0b2&os=android 12:36:53 INFO - Submitting to https://bounceradmin.mozilla.com/api/location_add/ 12:36:53 INFO - Server response 12:36:53 INFO - /mobile/releases/49.0b2/android-api-15/:lang/fennec-49.0b2.:lang.android-arm.apk 12:36:53 INFO - android-x86 (android-x86): /mobile/releases/49.0b2/android-x86/:lang/fennec-49.0b2.:lang.android-i386.apk 12:36:53 INFO - retry: Calling _api_call with args: ('location_add/', {'path': '/mobile/releases/49.0b2/android-x86/:lang/fennec-49.0b2.:lang.android-i386.apk', 'product': 'Fennec-49.0b2', 'os': 'android-x86'}), kwargs: {}, attempt #1 12:36:53 INFO - POST data: path=%2Fmobile%2Freleases%2F49.0b2%2Fandroid-x86%2F%3Alang%2Ffennec-49.0b2.%3Alang.android-i386.apk&product=Fennec-49.0b2&os=android-x86 12:36:53 INFO - Submitting to https://bounceradmin.mozilla.com/api/location_add/ 12:36:54 INFO - Server response 12:36:54 INFO - /mobile/releases/49.0b2/android-x86/:lang/fennec-49.0b2.:lang.android-i386.apk 12:36:54 INFO - ##### 12:36:54 INFO - ##### Finished submit step (success) 12:36:54 INFO - ##### 12:36:54 INFO - Running post-run listener: copy_logs_to_upload_dir 12:36:54 INFO - Copying logs to upload dir... 12:36:54 INFO - mkdir: /builds/slave/rel-m-beta-bncr_sub-0000000000/build/upload/logs 12:36:54 INFO - Copying logs to upload dir... program finished with exit code 0 elapsedTime=2.570496 ========= master_lag: 0.01 ========= ========= Finished 'scripts/scripts/bouncer_submitter.py -c ...' (results: 0, elapsed: 2 secs) (at 2016-08-09 12:36:54.359554) ========= ========= Started no change (results: 0, elapsed: 0 secs) (at 2016-08-09 12:36:54.360582) ========= bash -c 'for file in `ls -1`; do cat $file; done' in dir /builds/slave/rel-m-beta-bncr_sub-0000000000/properties (timeout 1200 secs) watching logfiles {} argv: ['bash', '-c', 'for file in `ls -1`; do cat $file; done'] environment: BUILDBOTCUSTOM=https://hg.mozilla.org/build/buildbotcustom BUILDBOT_CONFIGS=https://hg.mozilla.org/build/buildbot-configs CCACHE_HASHDIR= CLOBBERER_URL=https://api.pub.build.mozilla.org/clobberer/forceclobber G_BROKEN_FILENAMES=1 HISTCONTROL=ignoredups HISTSIZE=1000 HOME=/home/cltbld HOSTNAME=bld-linux64-spot-433.build.releng.usw2.mozilla.com LANG=en_US.UTF-8 LESSOPEN=|/usr/bin/lesspipe.sh %s LOGNAME=cltbld MAIL=/var/spool/mail/cltbld PATH=/usr/local/bin:/usr/lib64/ccache:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/cltbld/bin PWD=/builds/slave/rel-m-beta-bncr_sub-0000000000/properties SHELL=/bin/bash SHLVL=1 TERM=linux TMOUT=86400 USER=cltbld _=/tools/buildbot/bin/python using PTY: False program finished with exit code 0 elapsedTime=0.005568 ========= master_lag: 0.08 ========= ========= Finished no change (results: 0, elapsed: 0 secs) (at 2016-08-09 12:36:54.446441) ========= ========= Started 'rm -f ...' (results: 0, elapsed: 0 secs) (at 2016-08-09 12:36:54.446741) ========= rm -f oauth.txt in dir /builds/slave/rel-m-beta-bncr_sub-0000000000/. (timeout 1200 secs) watching logfiles {} argv: ['rm', '-f', 'oauth.txt'] environment: BUILDBOTCUSTOM=https://hg.mozilla.org/build/buildbotcustom BUILDBOT_CONFIGS=https://hg.mozilla.org/build/buildbot-configs CCACHE_HASHDIR= CLOBBERER_URL=https://api.pub.build.mozilla.org/clobberer/forceclobber G_BROKEN_FILENAMES=1 HISTCONTROL=ignoredups HISTSIZE=1000 HOME=/home/cltbld HOSTNAME=bld-linux64-spot-433.build.releng.usw2.mozilla.com LANG=en_US.UTF-8 LESSOPEN=|/usr/bin/lesspipe.sh %s LOGNAME=cltbld MAIL=/var/spool/mail/cltbld PATH=/usr/local/bin:/usr/lib64/ccache:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/cltbld/bin PWD=/builds/slave/rel-m-beta-bncr_sub-0000000000 SHELL=/bin/bash SHLVL=1 TERM=linux TMOUT=86400 USER=cltbld _=/tools/buildbot/bin/python using PTY: False program finished with exit code 0 elapsedTime=0.003652 ========= master_lag: 0.05 ========= ========= Finished 'rm -f ...' (results: 0, elapsed: 0 secs) (at 2016-08-09 12:36:54.496867) ========= ========= Total master_lag: 0.23 =========