ACCRE R9 Cluster Quick and Dirty Status
Report generated at Mon Aug 3 03:23:01 PM CDT 2026
Problem Nodes
HOSTNAMES STATE TIMESTAMP REASON COMMENT
cn1443 drain 2026-07-28T17:17:33 Nobody - RT100558 - Nobody - RT100558 - Flapping network interface
cn1484 drain 2026-07-22T21:07:25 Nobody - RT100374 - Nobody - RT100374 - Troubleshoot nic
cn1508 drng 2026-08-02T11:02:58 goffta1 - na - updat goffta1 - na - update-pending
cn1563 drng 2026-08-03T12:07:57 Kill task failed (Jo Nobody - RT100619 - packet-loss, reboot and check
cn1595 drain* 2026-07-27T16:58:32 Troy - RT100528 - Th Troy - RT100528 - Thermal issues
cn1601 drng 2026-08-02T11:02:58 goffta1 - na - updat goffta1 - na - update-pending
cn1602 drng 2026-08-02T11:02:58 goffta1 - na - updat goffta1 - na - update-pending
cn1608 drain* 2026-08-03T15:12:11 goffta1 - na - updat goffta1 - na - update-running
cn1617 drng 2026-08-02T11:02:58 goffta1 - na - updat goffta1 - na - update-pending
cn1703 drng 2026-08-02T11:02:58 goffta1 - na - updat goffta1 - na - update-pending
cn1704 drng 2026-08-02T11:02:58 goffta1 - na - updat goffta1 - na - update-pending
cn1705 drng 2026-08-02T11:02:58 goffta1 - na - updat goffta1 - na - update-pending
cn1814 drain 2026-08-03T15:04:32 goffta1 - na - updat goffta1 - na - update-running
gpu0300 drain 2026-08-03T15:04:50 goffta1 - na - updat goffta1 - na - update-running
Queue Summary (Batch)
GROUP USER ACTIVE_JOBS ACTIVE_CORES PENDING_JOBS PENDING_CORES
-----------------------------------------------------------------------------------------
accre_guests 0 0 1 100
haojz 0 0 1 100
-----------------------------------------------------------------------------------------
atate_lab 4 32 3 24
greinepm 4 32 3 24
-----------------------------------------------------------------------------------------
beam_lab 26 416 4 64
zhuj29 26 416 4 64
-----------------------------------------------------------------------------------------
bias_group 1 64 0 0
biasds 1 64 0 0
-----------------------------------------------------------------------------------------
biostat_faculty 1 4 40 40
nianh1 1 4 40 40
-----------------------------------------------------------------------------------------
brg_cores 1 8 0 0
kandelr 1 8 0 0
-----------------------------------------------------------------------------------------
cgg 0 0 1 64
liy110 0 0 1 64
-----------------------------------------------------------------------------------------
cms 894 4491 336 808
cmslocal 154 1342 61 126
cmspilot 740 3149 275 682
-----------------------------------------------------------------------------------------
coxlab 1 25 0 0
blostf1 1 25 0 0
-----------------------------------------------------------------------------------------
cqs_si 0 0 6 10
chenarsw 0 0 4 8
goodfeam 0 0 1 1
nichocm5 0 0 1 1
-----------------------------------------------------------------------------------------
das_lab 1 1 0 0
shiltmh1 1 1 0 0
-----------------------------------------------------------------------------------------
davis_lab 0 0 1 16
bluejor 0 0 1 16
-----------------------------------------------------------------------------------------
doran_lab 0 0 1 8
hopee 0 0 1 8
-----------------------------------------------------------------------------------------
econ_faculty 0 0 1 16
urquic3 0 0 1 16
-----------------------------------------------------------------------------------------
econgrads 0 0 1 1
chenl40 0 0 1 1
-----------------------------------------------------------------------------------------
feng_lab 7 36 11 11
chenj96 1 30 0 0
jiangl1 6 6 11 11
-----------------------------------------------------------------------------------------
g_cohen_lab 0 0 2 8
wangg13 0 0 2 8
-----------------------------------------------------------------------------------------
g_giri_group 2 4 0 0
basnetb1 2 4 0 0
-----------------------------------------------------------------------------------------
gravity 19 40 182 491
ligo 19 40 182 491
-----------------------------------------------------------------------------------------
haslag_group 5 72 84 1336
haslagph 5 72 84 1336
-----------------------------------------------------------------------------------------
h_biostat_kang 0 0 420 420
yanb1 0 0 420 420
-----------------------------------------------------------------------------------------
h_biostat_student 1 10 244 313
blackmh2 0 0 50 100
liuk20 0 0 1 1
namy1 0 0 192 192
yangc16 0 0 1 20
yih4 1 10 0 0
-----------------------------------------------------------------------------------------
h_ccm 1 4 0 0
boydb5 1 4 0 0
-----------------------------------------------------------------------------------------
h_cqs 393 498 31 496
wangy89 7 112 31 496
yanb1 386 386 0 0
-----------------------------------------------------------------------------------------
h_darby_lab 66 132 22 44
mills37 66 132 22 44
-----------------------------------------------------------------------------------------
hodges_lab 1 32 1 1
libbyj 0 0 1 1
torresev 1 32 0 0
-----------------------------------------------------------------------------------------
h_oguz_lab 1 8 1 8
wanj119 1 8 1 8
-----------------------------------------------------------------------------------------
h_vangard_1 27 162 0 0
wangj52 27 162 0 0
-----------------------------------------------------------------------------------------
isde-rer 0 0 2 16
champaca 0 0 1 8
maybergm 0 0 1 8
-----------------------------------------------------------------------------------------
karbstein_lab 1 8 0 0
saleemi 1 8 0 0
-----------------------------------------------------------------------------------------
kojetin_lab 1 4 1 7
arifovl 0 0 1 7
kojetid 1 4 0 0
-----------------------------------------------------------------------------------------
l2_bioinfo1 3 9 0 0
lyul1 3 9 0 0
-----------------------------------------------------------------------------------------
l3_aboud_lab 0 0 2 16
hongm1 0 0 2 16
-----------------------------------------------------------------------------------------
l3_bick_lab 1 16 0 0
nandamvk 1 16 0 0
-----------------------------------------------------------------------------------------
l3_runnoe_group 2 8 2 8
kaldorme 2 8 2 8
-----------------------------------------------------------------------------------------
l3_vuiis_cci 1 1 0 0
vuiis_daily_s 1 1 0 0
-----------------------------------------------------------------------------------------
l3_wilkey_lab 1 16 0 0
rubina4 1 16 0 0
-----------------------------------------------------------------------------------------
moro_lab 1 24 1 1
kitchc2 1 24 0 0
salams2 0 0 1 1
-----------------------------------------------------------------------------------------
nbody 0 0 1 256
smitm77 0 0 1 256
-----------------------------------------------------------------------------------------
neurogroup 1 5 0 0
roggeokk 1 5 0 0
-----------------------------------------------------------------------------------------
ng_lab 1 8 0 0
kimj119 1 8 0 0
-----------------------------------------------------------------------------------------
nodeupgrade 0 0 3 3
goffta1 0 0 3 3
-----------------------------------------------------------------------------------------
p_csb_meiler 1275 2337 7386 41204
agarwm5 0 0 6 387
holtcm 0 0 11 112
huntek1 857 857 2188 2188
moreljl 300 300 0 0
mothcw 0 0 1000 1000
tydingcw 118 1180 3704 37040
yange8 0 0 477 477
-----------------------------------------------------------------------------------------
p_dsi 0 0 8 12
yangi1 0 0 8 12
-----------------------------------------------------------------------------------------
p_englot_group 0 0 1 24
redaa1 0 0 1 24
-----------------------------------------------------------------------------------------
p_matheny_lab 2 2 0 0
ssemah2 2 2 0 0
-----------------------------------------------------------------------------------------
p_meiler 0 0 1 3
yange8 0 0 1 3
-----------------------------------------------------------------------------------------
rer 47 48 432 432
chey136 0 0 30 30
hum6 1 2 0 0
wonge7 46 46 402 402
-----------------------------------------------------------------------------------------
rke_group 7 7 0 0
maduren 7 7 0 0
-----------------------------------------------------------------------------------------
rokaslab 8 183 820 820
copea1 5 5 820 820
davidkt 2 130 0 0
riedlio 1 48 0 0
-----------------------------------------------------------------------------------------
schildcrout_lab 0 0 420 420
yanb1 0 0 420 420
-----------------------------------------------------------------------------------------
tk_lab 0 0 6 240
yoonh15 0 0 6 240
-----------------------------------------------------------------------------------------
vgi 2 9 0 0
parkj71 1 5 0 0
salerl1 1 4 0 0
-----------------------------------------------------------------------------------------
walker_lab 0 0 3 24
ravacm 0 0 3 24
-----------------------------------------------------------------------------------------
wankowicz_lab 56 56 2 2
wankows 56 56 2 2
-----------------------------------------------------------------------------------------
wan_lab 1 200 0 0
hardenn 1 200 0 0
-----------------------------------------------------------------------------------------
womelsdorf_lab 4 22 0 0
gerritcg 2 20 0 0
wenx1 2 2 0 0
-----------------------------------------------------------------------------------------
yang_lab 1 1 0 0
larkinsd 1 1 0 0
-----------------------------------------------------------------------------------------
yang_lab_csb 4 72 0 0
zhengm9 4 72 0 0
-----------------------------------------------------------------------------------------
Totals: 2872 9075 10484 47767
Queue Summary (Batch GPU)
GROUP USER ACTIVE_JOBS ACTIVE_GPUS PENDING_JOBS PENDING_GPUS
-----------------------------------------------------------------------------------------
accre_guests_acc 6 6 3 3
lib13 1 1 1 1
liy110 2 2 1 1
mcgrawke 0 0 1 1
wanj119 2 2 0 0
whitejt2 1 1 0 0
-----------------------------------------------------------------------------------------
cgg_acc 1 2 0 0
sandk13 1 2 0 0
-----------------------------------------------------------------------------------------
csb_gpu_acc 68 80 6812 6821
changga 48 48 1125 1125
gonzc11 0 0 5206 5206
huynhp 1 4 0 0
karadim 2 5 1 4
larkinsd 1 1 0 0
lindsr1 8 8 422 422
liuy145 4 4 54 54
ranx 1 4 0 0
yangz24 0 0 1 4
zhengm9 3 6 3 6
-----------------------------------------------------------------------------------------
h_oguz_lab_acc 2 2 1 1
wanj119 2 2 1 1
-----------------------------------------------------------------------------------------
maple_lab_acc 0 0 1 1
lif12 0 0 1 1
-----------------------------------------------------------------------------------------
nbody_acc 1 1 0 0
bustam1 1 1 0 0
-----------------------------------------------------------------------------------------
neurogroup_acc 2 2 12 12
jiangm5 2 2 12 12
-----------------------------------------------------------------------------------------
noble_lab_acc 2 2 8 8
vumq 2 2 8 8
-----------------------------------------------------------------------------------------
p_dsi_acc 0 0 20 20
yangi1 0 0 20 20
-----------------------------------------------------------------------------------------
p_meiler_acc 4 57 13 13
agarwm5 0 0 1 1
holtcm 0 0 2 2
kaot1 3 56 0 0
labeilro 1 1 0 0
moreljl 0 0 10 10
-----------------------------------------------------------------------------------------
taylor_group_acc 1 7 0 0
laaln1 1 7 0 0
-----------------------------------------------------------------------------------------
Totals: 87 159 6870 6879
Queue Summary (interactive)
GROUP USER ACTIVE_JOBS ACTIVE_CORES PENDING_JOBS PENDING_CORES
-----------------------------------------------------------------------------------------
booth_lab_int 1 2 0 0
comptoab 1 2 0 0
-----------------------------------------------------------------------------------------
h_choksi_lab_int 1 16 0 0
pilatjm 1 16 0 0
-----------------------------------------------------------------------------------------
maiziezhou_lab_int 1 4 98 392
tangk10 1 4 98 392
-----------------------------------------------------------------------------------------
maiziezhou_lab_phd_int 1 10 0 0
yuanw2 1 10 0 0
-----------------------------------------------------------------------------------------
nodeupgrade 0 0 2 2
goffta1 0 0 2 2
-----------------------------------------------------------------------------------------
rubinov_lab_int 2 26 0 0
mohamb2 1 16 0 0
sardarn 1 10 0 0
-----------------------------------------------------------------------------------------
yang_lab_int 2 32 0 0
shaoq1 2 32 0 0
-----------------------------------------------------------------------------------------
Totals: 8 90 100 394
Queue Summary (interactive_gpu)
GROUP USER ACTIVE_JOBS ACTIVE_GPUS PENDING_JOBS PENDING_GPUS
-----------------------------------------------------------------------------------------
accre_dgx_iacc 1 3 0 0
rubinom 1 3 0 0
-----------------------------------------------------------------------------------------
dsi_dgx_iacc 3 10 2 16
chattec 1 1 0 0
jenkm22 1 8 2 16
wut18 1 1 0 0
-----------------------------------------------------------------------------------------
p_matheny_lab_iacc 1 2 0 0
koolajd1 1 2 0 0
-----------------------------------------------------------------------------------------
Totals: 5 15 2 16
Partition Summary
PARTITION AVAIL TIMELIMIT NODES STATE NODELIST
interactive up 14-00:00:0 1 mix- cn1325
interactive up 14-00:00:0 1 drain cn1814
interactive up 14-00:00:0 4 mix cn[1287,1807,1812-1813]
interactive up 14-00:00:0 2 alloc cn[1326,1330]
interactive up 14-00:00:0 22 idle cn[1301-1302,1322-1324,1328-1329,1707,1800-1806,1808-1811,1815-1817]
batch* up 14-00:00:0 25 mix- cn[1379,1467,1469,1543,1597-1600,1603,1605,1607,1609,1616,1620,1625,1627,1631-1633,1701-1702,1706,1709-1710,2000]
batch* up 14-00:00:0 7 plnd cn[1604,1629-1630,1711-1714]
batch* up 14-00:00:0 2 drain* cn[1595,1608]
batch* up 14-00:00:0 8 drng cn[1508,1563,1601-1602,1617,1703-1705]
batch* up 14-00:00:0 2 drain cn[1443,1484]
batch* up 14-00:00:0 131 mix cn[1203,1205,1215,1217,1222-1224,1232,1234-1235,1238,1242,1257,1259-1260,1262,1264-1268,1272,1278,1280-1281,1288,1291-1292,1295-1296,1305-1310,1312,1315,1318,1320-1321,1331-1335,1337,1339-1342,1344-1347,1349-1350,1354,1357,1360-1361,1363-1365,1369,1372-1373,1376-1378,1381-1383,1385,1387-1392,1394,1397,1400,1405,1408,1414-1415,1419,1422-1423,1425-1427,1434,1445,1447-1448,1453-1454,1457,1460,1463,1473,1485,1490-1491,1494,1515,1517,1524,1527,1529-1531,1533,1544,1548,1550,1554-1556,1559,1573,1576,1583,1587,1593,1613-1614,1621,1708]
batch* up 14-00:00:0 215 alloc cn[1202,1204,1206-1213,1216,1218-1221,1226-1231,1233,1236-1237,1239-1241,1258,1261,1269-1271,1273-1275,1277,1279,1282-1286,1289-1290,1293-1294,1297-1299,1303-1304,1311,1313-1314,1316-1317,1327,1336,1338,1343,1348,1351-1353,1358-1359,1362,1366-1368,1371,1374-1375,1380,1384,1393,1395-1396,1398-1399,1401-1404,1406-1407,1409-1411,1416-1418,1420-1421,1424,1430-1432,1435-1439,1441-1442,1446,1449-1450,1452,1455-1456,1458,1461-1462,1464,1466,1468,1470-1472,1474-1483,1486-1489,1492-1493,1495-1500,1502-1507,1509-1514,1516,1518-1520,1522-1523,1525,1528,1532,1534-1538,1545-1547,1549,1551-1553,1557-1558,1561-1562,1564-1571,1574-1575,1577-1582,1584-1586,1588-1589,1592,1594,1596,1606,1610,1612,1615,1618-1619,1622-1624,1626,1628,1700]
batch* up 14-00:00:0 1 idle cn1501
batch_gpu up 14-00:00:0 1 drain gpu0300
batch_gpu up 14-00:00:0 33 mix gpu[0059,0062-0073,0075-0080,0082,0208,0301,0304-0310],gracehopper[01-02],hgx[02-03]
batch_gpu up 14-00:00:0 2 alloc gpu[0074,0081]
batch_gpu up 14-00:00:0 3 idle gpu[0085,0302-0303]
interactive_gpu up 14-00:00:0 4 mix dgx[01,03-04],gpu0207
interactive_gpu up 14-00:00:0 3 idle gpu[0058,0060-0061]
sam up 2-02:00:00 1 alloc cms-sam-01
sam up 2-02:00:00 1 idle cms-sam-02
reserved inact infinite 0 n/a