HammerCloud | ATLAS rss

Administration
<< Back

PanDA incidents history

The following table summarizes the incidents happened in HammerCloud tests. You can filter the results using the form below and clicking in 'Refresh'.


Incidents for last 72 hours

TimestampSeverityCommentTest
Sat 17 May 2025 04:06:02warningThe following online (non-HIMEM) sites need more test jobs for template 1294: UKI-SOUTHGRID-SUSX_TEST 20316937 »
Sat 17 May 2025 04:06:01warningUM6P (test): WhiteListing policy Last-Two-From-All not passed. See jobs ** http://bigpanda.cern.ch/job?pandaid=6654484773 ==> modificationHost: aipanda403.cern.ch pilot:::9000 Unknown error ** http://bigpanda.cern.ch/job?pandaid=6654315885 ==> modificationHost: aipanda403.cern.ch pilot:::9000 Unknown error 20316937 »
Sat 17 May 2025 04:06:01warningUM6P.UM6P: WhiteListing policy Last-Two-From-All not passed. See jobs ** http://bigpanda.cern.ch/job?pandaid=6654484773 ==> modificationHost: aipanda403.cern.ch pilot:::9000 Unknown error ** http://bigpanda.cern.ch/job?pandaid=6654315885 ==> modificationHost: aipanda403.cern.ch pilot:::9000 Unknown error 20316937 »
Sat 17 May 2025 04:06:01warningAM-01-AANL (test): WhiteListing policy Last-Two-From-All not passed. See jobs ** http://bigpanda.cern.ch/job?pandaid=6654632282 ==> modificationHost: aipanda403.cern.ch pilot:::9000 Unknown error ** http://bigpanda.cern.ch/job?pandaid=6654532748 ==> modificationHost: aipanda403.cern.ch pilot:::9000 Unknown error 20316937 »
Sat 17 May 2025 04:06:01warningAM-01-AANL.AM-01-AANL: WhiteListing policy Last-Two-From-All not passed. See jobs ** http://bigpanda.cern.ch/job?pandaid=6654632282 ==> modificationHost: aipanda403.cern.ch pilot:::9000 Unknown error ** http://bigpanda.cern.ch/job?pandaid=6654532748 ==> modificationHost: aipanda403.cern.ch pilot:::9000 Unknown error 20316937 »
Sat 17 May 2025 04:06:01blacklistingChecking these sites currently test because of HC: [u'AM-01-AANL', u'UM6P'] 20316937 »
Sat 17 May 2025 04:05:55blacklistingChecking these sites currently test because of HC: [] 20316932 »
Sat 17 May 2025 04:05:46warningThe following online sites need more test jobs for template 1214: UKI-LT2-QMUL_GPU 20316929 »
Sat 17 May 2025 04:05:45warningUKI-SCOTGRID-GLASGOW_GPU (test): WhiteListing policy Last-Two-From-All not passed. See jobs ** http://bigpanda.cern.ch/job?pandaid=6654612396 ==> modificationHost: slot1@gpu-d22-001.beowulf.cluster pilot:::1305 Failed to execute payload:There are no GPU devices available or correctly configured on this host. ddm:::200 Could not add files to DDM: Details: One of the PFNs provided {'davs://cephc08.gla.scotgrid.ac.uk:1094/atlas:datadisk/rucio/hc_test/a2/46/4af53c92-fc6d-4396-8b6a-d2acae553fea_93327.1.job.log.tgz'} for [{'scope': 'hc_test', 'name': '4af53c92-fc6d-4396-8b6a-d2acae553fea_93327.1.job.log.tgz'}] does not match the Rucio expected PFNs: {'davs://cephc08.gla.scotgrid.ac.uk:1094/atlas:scratchdisk/rucio/hc_test/a2/46/4af53c92-fc6d-4396-8b6a-d2acae553fea_93327.1.job.log.tgz', 'davs://cephc07.gla.scotgrid.a ** http://bigpanda.cern.ch/job?pandaid=6654609149 ==> modificationHost: slot1@gpu-d22-001.beowulf.cluster pilot:::1305 Failed to execute payload:There are no GPU devices available or correctly configured on this host. ddm:::200 Could not add files to DDM: Details: One of the PFNs provided {'davs://cephc08.gla.scotgrid.ac.uk:1094/atlas:datadisk/rucio/hc_test/1a/fd/f80b61c3-7cd8-4291-8bd4-1fc5ca24fbb8_54881.1.job.log.tgz'} for [{'scope': 'hc_test', 'name': 'f80b61c3-7cd8-4291-8bd4-1fc5ca24fbb8_54881.1.job.log.tgz'}] does not match the Rucio expected PFNs: {'davs://cephc08.gla.scotgrid.ac.uk:1094/atlas:scratchdisk/rucio/hc_test/1a/fd/f80b61c3-7cd8-4291-8bd4-1fc5ca24fbb8_54881.1.job.log.tgz', 'davs://cephc07.gla.scotgrid.a 20316929 »
Sat 17 May 2025 04:05:45warningUKI-SCOTGRID-GLASGOW_CEPH.UKI-SCOTGRID-GLASGOW_GPU: WhiteListing policy Last-Two-From-All not passed. See jobs ** http://bigpanda.cern.ch/job?pandaid=6654612396 ==> modificationHost: slot1@gpu-d22-001.beowulf.cluster pilot:::1305 Failed to execute payload:There are no GPU devices available or correctly configured on this host. ddm:::200 Could not add files to DDM: Details: One of the PFNs provided {'davs://cephc08.gla.scotgrid.ac.uk:1094/atlas:datadisk/rucio/hc_test/a2/46/4af53c92-fc6d-4396-8b6a-d2acae553fea_93327.1.job.log.tgz'} for [{'scope': 'hc_test', 'name': '4af53c92-fc6d-4396-8b6a-d2acae553fea_93327.1.job.log.tgz'}] does not match the Rucio expected PFNs: {'davs://cephc08.gla.scotgrid.ac.uk:1094/atlas:scratchdisk/rucio/hc_test/a2/46/4af53c92-fc6d-4396-8b6a-d2acae553fea_93327.1.job.log.tgz', 'davs://cephc07.gla.scotgrid.a ** http://bigpanda.cern.ch/job?pandaid=6654609149 ==> modificationHost: slot1@gpu-d22-001.beowulf.cluster pilot:::1305 Failed to execute payload:There are no GPU devices available or correctly configured on this host. ddm:::200 Could not add files to DDM: Details: One of the PFNs provided {'davs://cephc08.gla.scotgrid.ac.uk:1094/atlas:datadisk/rucio/hc_test/1a/fd/f80b61c3-7cd8-4291-8bd4-1fc5ca24fbb8_54881.1.job.log.tgz'} for [{'scope': 'hc_test', 'name': 'f80b61c3-7cd8-4291-8bd4-1fc5ca24fbb8_54881.1.job.log.tgz'}] does not match the Rucio expected PFNs: {'davs://cephc08.gla.scotgrid.ac.uk:1094/atlas:scratchdisk/rucio/hc_test/1a/fd/f80b61c3-7cd8-4291-8bd4-1fc5ca24fbb8_54881.1.job.log.tgz', 'davs://cephc07.gla.scotgrid.a 20316929 »
Sat 17 May 2025 04:05:45blacklistingChecking these sites currently test because of HC: [u'UKI-SCOTGRID-GLASGOW_GPU'] 20316929 »
Sat 17 May 2025 04:05:39warningThe following online (non-MCORE/non-HIMEM) sites need more test jobs for template 1289: AU-Melbourne, CSCS-LCG2-ALPS, CYFRONET_EOS, DESY-ZN, GRIF-IRFU, INFN-NAPOLI-ATLAS, MPPMU, SAMPA, SARA-MATRIX, UKI-NORTHGRID-LIV-HEP, UKI-NORTHGRID-MAN-HEP, UNI-FREIBURG_NHR_TEST, UNIGE-BAOBAB, praguelcg2 20316946 »
Sat 17 May 2025 04:05:39warningThe following online (non-MCORE/non-HIMEM) sites need more test jobs for template 1293: MPPMU, SAMPA 20316946 »
Sat 17 May 2025 04:05:31warningNIKHEF (test): WhiteListing policy Last-Two-From-All not passed. See jobs ** http://bigpanda.cern.ch/job?pandaid=6654607780 ==> modificationHost: wn-snel-024.farm.nikhef.nl pilot:::1099 Failed to stage-in file: [ReadTimeout(ReadTimeoutError("HTTPSConnectionPool(host='atlas-rucio-auth.cern.ch', port=443): Read timed out. (read timeout=600)"))]:failed to transfer files using copytools=['rucio'] ** http://bigpanda.cern.ch/job?pandaid=6654684167 ==> modificationHost: wn-choc-006.farm.nikhef.nl pilot:::1137 Failed to stage-out file: hc_test:output.1.24abfbce-4738-47d8-839d-dfcb638b91fa_5931.pool.root from NIKHEF_DATADISK, No protocol for provided settings found : {'availability_delete': True, 'availability_read': True, 'availability_write': True, 'credentials': None, 'deterministic': True, 'domain': ['lan', WhiteListing policy Last-Two-From-All not passed. See jobs ** http://bigpanda.cern.ch/job?pandaid=6654737651 ==> modificationHost: wn-sate-050.farm.nikhef.nl pilot:::1152 File transfer timed out during stage-out: hc_test:output.1.c60017c6-da27-4e2e-8578-b1fdc5fee792_12844.pool.root to NIKHEF_SCRATCHDISK, copy command timed out: TimeoutException: Timeout reached, timeout=310 seconds')]:failed to transfer files using copytools=['rucio'] ** http://bigpanda.cern.ch/job?pandaid=6654689685 ==> modificationHost: wn-snel-025.farm.nikhef.nl pilot:::1361 Remote file could not be opened:Remote file(s) could not be opened: ['root://atlas.dcache.nikhef.nl:1094//pnfs/nikhef.nl/data/atlas/atlasdatadisk/rucio/mc20_13TeV/ff/63/DAOD_PHYS.34870879._000001.pool.root.1'] 20316946 »
Sat 17 May 2025 04:05:31warningNIKHEF.NIKHEF: WhiteListing policy Last-Two-From-All not passed. See jobs ** http://bigpanda.cern.ch/job?pandaid=6654607780 ==> modificationHost: wn-snel-024.farm.nikhef.nl pilot:::1099 Failed to stage-in file: [ReadTimeout(ReadTimeoutError("HTTPSConnectionPool(host='atlas-rucio-auth.cern.ch', port=443): Read timed out. (read timeout=600)"))]:failed to transfer files using copytools=['rucio'] ** http://bigpanda.cern.ch/job?pandaid=6654684167 ==> modificationHost: wn-choc-006.farm.nikhef.nl pilot:::1137 Failed to stage-out file: hc_test:output.1.24abfbce-4738-47d8-839d-dfcb638b91fa_5931.pool.root from NIKHEF_DATADISK, No protocol for provided settings found : {'availability_delete': True, 'availability_read': True, 'availability_write': True, 'credentials': None, 'deterministic': True, 'domain': ['lan', 20316946 »
Sat 17 May 2025 04:05:30blacklistingChecking these sites currently test because of HC: [u'NIKHEF'] 20316946 »
Sat 17 May 2025 04:05:30warningThe following online (non-MCORE/non-HIMEM) sites need more test jobs for template 1272: MPPMU, SAMPA, UNIGE-BAOBAB 20316954 »
Sat 17 May 2025 04:05:30warningThe following online (non-MCORE/non-HIMEM) sites need more test jobs for template 1273: CSCS-LCG2-ALPS, MPPMU, SAMPA, UNIGE-BAOBAB 20316954 »
Sat 17 May 2025 04:05:18warning** New online sites not in templates: [u'BNL_PROD_INTEL', u'EMMY_DESY'] 20316954 »
Sat 17 May 2025 04:05:18warningGoeGrid.EMMY_DESY: Site EMMY_DESY missing from some templates: [1273L, 1272L]. Resource configuration: ** name: EMMY_DESY ** master_resource: GoeGrid ** type: unified ** capability: ucore ** is_default: False ** hc_param: AutoExclusion 20316954 »
Sat 17 May 2025 04:05:18warningBNL.BNL_PROD_INTEL: Site BNL_PROD_INTEL missing from some templates: [1273L, 1272L]. Resource configuration: ** name: BNL_PROD_INTEL ** master_resource: BNL ** type: unified ** capability: ucore ** is_default: False ** hc_param: AutoExclusion 20316954 »
Sat 17 May 2025 04:05:18warningNIKHEF (test): WhiteListing policy Last-Two-From-All not passed. See jobs ** http://bigpanda.cern.ch/job?pandaid=6654737651 ==> modificationHost: wn-sate-050.farm.nikhef.nl pilot:::1152 File transfer timed out during stage-out: hc_test:output.1.c60017c6-da27-4e2e-8578-b1fdc5fee792_12844.pool.root to NIKHEF_SCRATCHDISK, copy command timed out: TimeoutException: Timeout reached, timeout=310 seconds')]:failed to transfer files using copytools=['rucio'] ** http://bigpanda.cern.ch/job?pandaid=6654689685 ==> modificationHost: wn-snel-025.farm.nikhef.nl pilot:::1361 Remote file could not be opened:Remote file(s) could not be opened: ['root://atlas.dcache.nikhef.nl:1094//pnfs/nikhef.nl/data/atlas/atlasdatadisk/rucio/mc20_13TeV/ff/63/DAOD_PHYS.34870879._000001.pool.root.1'] 20316954 »
Sat 17 May 2025 04:05:18warningNIKHEF.NIKHEF: WhiteListing policy Last-Two-From-All not passed. See jobs ** http://bigpanda.cern.ch/job?pandaid=6654737651 ==> modificationHost: wn-sate-050.farm.nikhef.nl pilot:::1152 File transfer timed out during stage-out: hc_test:output.1.c60017c6-da27-4e2e-8578-b1fdc5fee792_12844.pool.root to NIKHEF_SCRATCHDISK, copy command timed out: TimeoutException: Timeout reached, timeout=310 seconds')]:failed to transfer files using copytools=['rucio'] ** http://bigpanda.cern.ch/job?pandaid=6654689685 ==> modificationHost: wn-snel-025.farm.nikhef.nl pilot:::1361 Remote file could not be opened:Remote file(s) could not be opened: ['root://atlas.dcache.nikhef.nl:1094//pnfs/nikhef.nl/data/atlas/atlasdatadisk/rucio/mc20_13TeV/ff/63/DAOD_PHYS.34870879._000001.pool.root.1'] 20316954 »
Sat 17 May 2025 04:05:17blacklistingChecking these sites currently test because of HC: [u'NIKHEF'] 20316954 »
Sat 17 May 2025 04:05:12warningThe following online (non-MCORE/non-HIMEM) sites need more test jobs for template 1289: CA-IAAS-T3, CERN-AZURE, DE-TARDIS, EELA-UTFSM, HONGKONG, LRZ-LMU_TEST, NCG-INGRID-PT, PUHTI, TOKYO_CLOUD, UKI-SOUTHGRID-BHAM-HEP, UNI-SIEGEN-HEP 20316946 »
«« first Page 1 of 519 next » last »»