请提供下述完整信息以便快速定位问题/Please provide the following information to quickly locate the problem
-
系统环境/System Environment:Ubuntu 18.04.5
-
版本号/Version:Paddle:gpu-2.3.0.post112 PaddleOCR:2.6.0.1
-
问题相关组件/Related components: 端到端弯曲印章中文文本识别-
-
运行指令/Command Code:python tools/train.py -c configs/e2e/e2e_r50_vd_pg.yml
-
完整报错/Complete Error Message:
[2022/10/13 08:11:45] ppocr INFO: cur metric, total_num_gt: 5047, total_num_det: 20, global_accumulative_recall: 0, hit_str_count: 0, recall: 0.0, precision: 0.0, f_score: 0, seqerr: 1, recall_e2e: 0.0, precision_e2e: 0.0, f_score_e2e: 0, fps: 25.780213590307252 [2022/10/13 08:11:45] ppocr INFO: save best model is to ./output/pgnet_r50_vd_totaltext/best_accuracy [2022/10/13 08:11:45] ppocr INFO: best metric, f_score_e2e: 0, is_float16: False, total_num_gt: 5047, total_num_det: 20, global_accumulative_recall: 0, hit_str_count: 0, recall: 0.0, precision: 0.0, f_score: 0, seqerr: 1, recall_e2e: 0.0, precision_e2e: 0.0, fps: 25.780213590307252, best_epoch: 21, start_epoch: 2 [2022/10/13 08:11:47] ppocr INFO: epoch: [21/600], global_step: 39010, lr: 0.000500, loss: 11.084884, score_loss: 0.086298, border_loss: 0.046290, direction_loss: 0.043400, ctc_loss: 2.181674, avg_reader_cost: 0.00012 s, avg_batch_cost: 0.20609 s, avg_samples: 4.0, ips: 19.40912 samples/s, eta: 3 days, 18:45:29 [2022/10/13 08:11:50] ppocr INFO: epoch: [21/600], global_step: 39020, lr: 0.000500, loss: 13.444241, score_loss: 0.084993, border_loss: 0.044626, direction_loss: 0.045938, ctc_loss: 2.651674, avg_reader_cost: 0.00011 s, avg_batch_cost: 0.20659 s, avg_samples: 4.0, ips: 19.36160 samples/s, eta: 3 days, 18:45:03 [2022/10/13 08:11:52] ppocr INFO: epoch: [21/600], global_step: 39030, lr: 0.000500, loss: 14.512180, score_loss: 0.090808, border_loss: 0.045525, direction_loss: 0.045752, ctc_loss: 2.857359, avg_reader_cost: 0.00011 s, avg_batch_cost: 0.20597 s, avg_samples: 4.0, ips: 19.41998 samples/s, eta: 3 days, 18:44:38 [2022/10/13 08:11:54] ppocr INFO: epoch: [21/600], global_step: 39040, lr: 0.000500, loss: 15.092965, score_loss: 0.090317, border_loss: 0.045396, direction_loss: 0.041555, ctc_loss: 2.976824, avg_reader_cost: 0.00011 s, avg_batch_cost: 0.20700 s, avg_samples: 4.0, ips: 19.32373 samples/s, eta: 3 days, 18:44:13 [2022/10/13 08:11:56] ppocr INFO: epoch: [21/600], global_step: 39050, lr: 0.000500, loss: 16.404400, score_loss: 0.087255, border_loss: 0.045163, direction_loss: 0.040814, ctc_loss: 3.247627, avg_reader_cost: 0.00011 s, avg_batch_cost: 0.20663 s, avg_samples: 4.0, ips: 19.35795 samples/s, eta: 3 days, 18:43:48 [2022/10/13 08:11:58] ppocr INFO: epoch: [21/600], global_step: 39060, lr: 0.000500, loss: 12.867165, score_loss: 0.086150, border_loss: 0.044533, direction_loss: 0.043478, ctc_loss: 2.531326, avg_reader_cost: 0.00011 s, avg_batch_cost: 0.20685 s, avg_samples: 4.0, ips: 19.33762 samples/s, eta: 3 days, 18:43:23 [2022/10/13 08:12:00] ppocr INFO: epoch: [21/600], global_step: 39070, lr: 0.000500, loss: 17.487686, score_loss: 0.089322, border_loss: 0.049033, direction_loss: 0.046898, ctc_loss: 3.463358, avg_reader_cost: 0.00010 s, avg_batch_cost: 0.20684 s, avg_samples: 4.0, ips: 19.33858 samples/s, eta: 3 days, 18:42:58 [2022/10/13 08:12:02] ppocr INFO: epoch: [21/600], global_step: 39080, lr: 0.000500, loss: 15.278533, score_loss: 0.087102, border_loss: 0.050207, direction_loss: 0.046234, ctc_loss: 3.016279, avg_reader_cost: 0.00011 s, avg_batch_cost: 0.20691 s, avg_samples: 4.0, ips: 19.33216 samples/s, eta: 3 days, 18:42:33 [2022/10/13 08:12:04] ppocr INFO: epoch: [21/600], global_step: 39090, lr: 0.000500, loss: 14.819781, score_loss: 0.083840, border_loss: 0.044131, direction_loss: 0.042342, ctc_loss: 2.932538, avg_reader_cost: 0.00011 s, avg_batch_cost: 0.20678 s, avg_samples: 4.0, ips: 19.34463 samples/s, eta: 3 days, 18:42:08 [2022/10/13 08:12:06] ppocr INFO: epoch: [21/600], global_step: 39100, lr: 0.000500, loss: 18.800060, score_loss: 0.082650, border_loss: 0.040598, direction_loss: 0.041834, ctc_loss: 3.715833, avg_reader_cost: 0.00011 s, avg_batch_cost: 0.20700 s, avg_samples: 4.0, ips: 19.32332 samples/s, eta: 3 days, 18:41:43 [2022/10/13 08:12:08] ppocr INFO: epoch: [21/600], global_step: 39110, lr: 0.000500, loss: 17.031275, score_loss: 0.085180, border_loss: 0.041527, direction_loss: 0.040754, ctc_loss: 3.362732, avg_reader_cost: 0.00010 s, avg_batch_cost: 0.20672 s, avg_samples: 4.0, ips: 19.34979 samples/s, eta: 3 days, 18:41:18 [2022/10/13 08:12:11] ppocr INFO: epoch: [21/600], global_step: 39120, lr: 0.000500, loss: 15.832161, score_loss: 0.082785, border_loss: 0.043618, direction_loss: 0.040197, ctc_loss: 3.134106, avg_reader_cost: 0.00010 s, avg_batch_cost: 0.20749 s, avg_samples: 4.0, ips: 19.27808 samples/s, eta: 3 days, 18:40:53 [2022/10/13 08:12:13] ppocr INFO: epoch: [21/600], global_step: 39130, lr: 0.000500, loss: 15.942187, score_loss: 0.087203, border_loss: 0.050523, direction_loss: 0.046406, ctc_loss: 3.152855, avg_reader_cost: 0.00010 s, avg_batch_cost: 0.20685 s, avg_samples: 4.0, ips: 19.33738 samples/s, eta: 3 days, 18:40:28 [2022/10/13 08:12:15] ppocr INFO: epoch: [21/600], global_step: 39140, lr: 0.000500, loss: 14.323619, score_loss: 0.082806, border_loss: 0.047834, direction_loss: 0.043380, ctc_loss: 2.826660, avg_reader_cost: 0.00010 s, avg_batch_cost: 0.20715 s, avg_samples: 4.0, ips: 19.30932 samples/s, eta: 3 days, 18:40:03 [2022/10/13 08:12:17] ppocr INFO: epoch: [21/600], global_step: 39150, lr: 0.000500, loss: 15.638828, score_loss: 0.081104, border_loss: 0.040084, direction_loss: 0.041608, ctc_loss: 3.096221, avg_reader_cost: 0.00011 s, avg_batch_cost: 0.20678 s, avg_samples: 4.0, ips: 19.34413 samples/s, eta: 3 days, 18:39:38 [2022/10/13 08:12:19] ppocr INFO: epoch: [21/600], global_step: 39160, lr: 0.000500, loss: 14.928933, score_loss: 0.085013, border_loss: 0.044312, direction_loss: 0.044478, ctc_loss: 2.951595, avg_reader_cost: 0.00010 s, avg_batch_cost: 0.20727 s, avg_samples: 4.0, ips: 19.29857 samples/s, eta: 3 days, 18:39:13 [2022/10/13 08:12:21] ppocr INFO: epoch: [21/600], global_step: 39170, lr: 0.000500, loss: 15.088455, score_loss: 0.081936, border_loss: 0.044755, direction_loss: 0.044702, ctc_loss: 2.985130, avg_reader_cost: 0.00010 s, avg_batch_cost: 0.20697 s, avg_samples: 4.0, ips: 19.32622 samples/s, eta: 3 days, 18:38:48 [2022/10/13 08:12:23] ppocr INFO: epoch: [21/600], global_step: 39180, lr: 0.000500, loss: 14.076549, score_loss: 0.082387, border_loss: 0.044635, direction_loss: 0.042108, ctc_loss: 2.780841, avg_reader_cost: 0.00011 s, avg_batch_cost: 0.20742 s, avg_samples: 4.0, ips: 19.28475 samples/s, eta: 3 days, 18:38:24 [2022/10/13 08:12:25] ppocr INFO: epoch: [21/600], global_step: 39190, lr: 0.000500, loss: 12.259347, score_loss: 0.080176, border_loss: 0.041576, direction_loss: 0.041219, ctc_loss: 2.420116, avg_reader_cost: 0.00011 s, avg_batch_cost: 0.20713 s, avg_samples: 4.0, ips: 19.31112 samples/s, eta: 3 days, 18:37:59 [2022/10/13 08:12:27] ppocr INFO: epoch: [21/600], global_step: 39200, lr: 0.000500, loss: 13.625978, score_loss: 0.082946, border_loss: 0.042645, direction_loss: 0.041504, ctc_loss: 2.695551, avg_reader_cost: 0.00011 s, avg_batch_cost: 0.20713 s, avg_samples: 4.0, ips: 19.31185 samples/s, eta: 3 days, 18:37:34 [2022/10/13 08:12:30] ppocr INFO: epoch: [21/600], global_step: 39210, lr: 0.000500, loss: 14.726393, score_loss: 0.088021, border_loss: 0.047623, direction_loss: 0.046091, ctc_loss: 2.907197, avg_reader_cost: 0.00012 s, avg_batch_cost: 0.20721 s, avg_samples: 4.0, ips: 19.30391 samples/s, eta: 3 days, 18:37:09 [2022/10/13 08:12:32] ppocr INFO: epoch: [21/600], global_step: 39220, lr: 0.000500, loss: 13.920656, score_loss: 0.090113, border_loss: 0.046403, direction_loss: 0.045524, ctc_loss: 2.751204, avg_reader_cost: 0.00010 s, avg_batch_cost: 0.20745 s, avg_samples: 4.0, ips: 19.28216 samples/s, eta: 3 days, 18:36:44 [2022/10/13 08:12:34] ppocr INFO: epoch: [21/600], global_step: 39230, lr: 0.000500, loss: 13.388535, score_loss: 0.084454, border_loss: 0.045113, direction_loss: 0.042603, ctc_loss: 2.636041, avg_reader_cost: 0.00011 s, avg_batch_cost: 0.20767 s, avg_samples: 4.0, ips: 19.26173 samples/s, eta: 3 days, 18:36:20 [2022/10/13 08:12:36] ppocr INFO: epoch: [21/600], global_step: 39240, lr: 0.000500, loss: 13.067587, score_loss: 0.086492, border_loss: 0.046417, direction_loss: 0.041737, ctc_loss: 2.576696, avg_reader_cost: 0.00010 s, avg_batch_cost: 0.20745 s, avg_samples: 4.0, ips: 19.28201 samples/s, eta: 3 days, 18:35:55 [2022/10/13 08:12:38] ppocr INFO: epoch: [21/600], global_step: 39250, lr: 0.000500, loss: 14.064493, score_loss: 0.089276, border_loss: 0.047259, direction_loss: 0.041287, ctc_loss: 2.780887, avg_reader_cost: 0.00010 s, avg_batch_cost: 0.20747 s, avg_samples: 4.0, ips: 19.28007 samples/s, eta: 3 days, 18:35:30 [2022/10/13 08:12:40] ppocr INFO: epoch: [21/600], global_step: 39260, lr: 0.000500, loss: 15.320513, score_loss: 0.085148, border_loss: 0.044710, direction_loss: 0.041958, ctc_loss: 3.031088, avg_reader_cost: 0.00010 s, avg_batch_cost: 0.20773 s, avg_samples: 4.0, ips: 19.25603 samples/s, eta: 3 days, 18:35:06 [2022/10/13 08:12:42] ppocr INFO: epoch: [21/600], global_step: 39270, lr: 0.000500, loss: 15.320513, score_loss: 0.084727, border_loss: 0.043460, direction_loss: 0.042324, ctc_loss: 3.031088, avg_reader_cost: 0.00010 s, avg_batch_cost: 0.20768 s, avg_samples: 4.0, ips: 19.26054 samples/s, eta: 3 days, 18:34:41 [2022/10/13 08:12:44] ppocr INFO: epoch: [21/600], global_step: 39280, lr: 0.000500, loss: 16.668587, score_loss: 0.091848, border_loss: 0.049249, direction_loss: 0.046950, ctc_loss: 3.290785, avg_reader_cost: 0.00011 s, avg_batch_cost: 0.20759 s, avg_samples: 4.0, ips: 19.26921 samples/s, eta: 3 days, 18:34:17 [2022/10/13 08:12:47] ppocr INFO: epoch: [21/600], global_step: 39290, lr: 0.000500, loss: 16.152910, score_loss: 0.082048, border_loss: 0.045222, direction_loss: 0.039182, ctc_loss: 3.194974, avg_reader_cost: 0.00011 s, avg_batch_cost: 0.20783 s, avg_samples: 4.0, ips: 19.24613 samples/s, eta: 3 days, 18:33:52 [2022/10/13 08:12:49] ppocr INFO: epoch: [21/600], global_step: 39300, lr: 0.000500, loss: 13.718008, score_loss: 0.081055, border_loss: 0.038359, direction_loss: 0.037191, ctc_loss: 2.710433, avg_reader_cost: 0.00010 s, avg_batch_cost: 0.20829 s, avg_samples: 4.0, ips: 19.20398 samples/s, eta: 3 days, 18:33:28 [2022/10/13 08:12:51] ppocr INFO: epoch: [21/600], global_step: 39310, lr: 0.000500, loss: 17.991924, score_loss: 0.081055, border_loss: 0.040344, direction_loss: 0.038653, ctc_loss: 3.555475, avg_reader_cost: 0.00011 s, avg_batch_cost: 0.20725 s, avg_samples: 4.0, ips: 19.30019 samples/s, eta: 3 days, 18:33:03 [2022/10/13 08:12:53] ppocr INFO: epoch: [21/600], global_step: 39320, lr: 0.000500, loss: 17.592228, score_loss: 0.081578, border_loss: 0.040199, direction_loss: 0.037187, ctc_loss: 3.474288, avg_reader_cost: 0.00010 s, avg_batch_cost: 0.20742 s, avg_samples: 4.0, ips: 19.28488 samples/s, eta: 3 days, 18:32:39 [2022/10/13 08:12:55] ppocr INFO: epoch: [21/600], global_step: 39330, lr: 0.000500, loss: 17.592228, score_loss: 0.086177, border_loss: 0.044134, direction_loss: 0.041469, ctc_loss: 3.474288, avg_reader_cost: 0.00010 s, avg_batch_cost: 0.20782 s, avg_samples: 4.0, ips: 19.24776 samples/s, eta: 3 days, 18:32:14 [2022/10/13 08:12:57] ppocr INFO: epoch: [21/600], global_step: 39340, lr: 0.000500, loss: 16.516041, score_loss: 0.084742, border_loss: 0.044347, direction_loss: 0.041895, ctc_loss: 3.269265, avg_reader_cost: 0.00010 s, avg_batch_cost: 0.20784 s, avg_samples: 4.0, ips: 19.24522 samples/s, eta: 3 days, 18:31:50 [2022/10/13 08:12:59] ppocr INFO: epoch: [21/600], global_step: 39350, lr: 0.000500, loss: 16.344032, score_loss: 0.080384, border_loss: 0.038532, direction_loss: 0.038218, ctc_loss: 3.234766, avg_reader_cost: 0.00010 s, avg_batch_cost: 0.20747 s, avg_samples: 4.0, ips: 19.27956 samples/s, eta: 3 days, 18:31:25 [2022/10/13 08:13:01] ppocr INFO: epoch: [21/600], global_step: 39360, lr: 0.000500, loss: 17.034090, score_loss: 0.081750, border_loss: 0.039743, direction_loss: 0.037217, ctc_loss: 3.370844, avg_reader_cost: 0.00010 s, avg_batch_cost: 0.20809 s, avg_samples: 4.0, ips: 19.22280 samples/s, eta: 3 days, 18:31:01 [2022/10/13 08:13:03] ppocr INFO: epoch: [21/600], global_step: 39370, lr: 0.000500, loss: 10.498444, score_loss: 0.082463, border_loss: 0.039515, direction_loss: 0.037831, ctc_loss: 2.065223, avg_reader_cost: 0.00010 s, avg_batch_cost: 0.20770 s, avg_samples: 4.0, ips: 19.25866 samples/s, eta: 3 days, 18:30:36 [2022/10/13 08:13:06] ppocr INFO: epoch: [21/600], global_step: 39380, lr: 0.000500, loss: 14.969946, score_loss: 0.085689, border_loss: 0.046408, direction_loss: 0.043892, ctc_loss: 2.964043, avg_reader_cost: 0.00010 s, avg_batch_cost: 0.20788 s, avg_samples: 4.0, ips: 19.24187 samples/s, eta: 3 days, 18:30:12 [2022/10/13 08:13:08] ppocr INFO: epoch: [21/600], global_step: 39390, lr: 0.000500, loss: 20.923199, score_loss: 0.091030, border_loss: 0.047799, direction_loss: 0.047457, ctc_loss: 4.137798, avg_reader_cost: 0.00010 s, avg_batch_cost: 0.20772 s, avg_samples: 4.0, ips: 19.25670 samples/s, eta: 3 days, 18:29:47 [2022/10/13 08:13:10] ppocr INFO: epoch: [21/600], global_step: 39400, lr: 0.000500, loss: 17.352938, score_loss: 0.084088, border_loss: 0.043067, direction_loss: 0.041650, ctc_loss: 3.436025, avg_reader_cost: 0.00010 s, avg_batch_cost: 0.20749 s, avg_samples: 4.0, ips: 19.27769 samples/s, eta: 3 days, 18:29:23 [2022/10/13 08:13:12] ppocr INFO: epoch: [21/600], global_step: 39410, lr: 0.000500, loss: 10.250685, score_loss: 0.084215, border_loss: 0.042672, direction_loss: 0.040468, ctc_loss: 2.019598, avg_reader_cost: 0.00010 s, avg_batch_cost: 0.20793 s, avg_samples: 4.0, ips: 19.23761 samples/s, eta: 3 days, 18:28:59 [2022/10/13 08:13:14] ppocr INFO: epoch: [21/600], global_step: 39420, lr: 0.000500, loss: 11.255610, score_loss: 0.085667, border_loss: 0.044883, direction_loss: 0.040183, ctc_loss: 2.221097, avg_reader_cost: 0.00011 s, avg_batch_cost: 0.20787 s, avg_samples: 4.0, ips: 19.24310 samples/s, eta: 3 days, 18:28:34 [2022/10/13 08:13:16] ppocr INFO: epoch: [21/600], global_step: 39430, lr: 0.000500, loss: 14.567810, score_loss: 0.085414, border_loss: 0.045642, direction_loss: 0.041206, ctc_loss: 2.879442, avg_reader_cost: 0.00010 s, avg_batch_cost: 0.20770 s, avg_samples: 4.0, ips: 19.25885 samples/s, eta: 3 days, 18:28:10 [2022/10/13 08:13:18] ppocr INFO: epoch: [21/600], global_step: 39440, lr: 0.000500, loss: 17.110062, score_loss: 0.088368, border_loss: 0.046972, direction_loss: 0.044832, ctc_loss: 3.387146, avg_reader_cost: 0.00011 s, avg_batch_cost: 0.20813 s, avg_samples: 4.0, ips: 19.21874 samples/s, eta: 3 days, 18:27:46 [2022/10/13 08:13:20] ppocr INFO: epoch: [21/600], global_step: 39450, lr: 0.000500, loss: 16.751495, score_loss: 0.090598, border_loss: 0.047180, direction_loss: 0.045162, ctc_loss: 3.314862, avg_reader_cost: 0.00011 s, avg_batch_cost: 0.20785 s, avg_samples: 4.0, ips: 19.24457 samples/s, eta: 3 days, 18:27:21 [2022/10/13 08:13:23] ppocr INFO: epoch: [21/600], global_step: 39460, lr: 0.000500, loss: 15.370590, score_loss: 0.086081, border_loss: 0.043122, direction_loss: 0.040073, ctc_loss: 3.037100, avg_reader_cost: 0.00010 s, avg_batch_cost: 0.20778 s, avg_samples: 4.0, ips: 19.25136 samples/s, eta: 3 days, 18:26:57 [2022/10/13 08:13:25] ppocr INFO: epoch: [21/600], global_step: 39470, lr: 0.000500, loss: 15.925407, score_loss: 0.084162, border_loss: 0.042518, direction_loss: 0.040073, ctc_loss: 3.149744, avg_reader_cost: 0.00010 s, avg_batch_cost: 0.20787 s, avg_samples: 4.0, ips: 19.24268 samples/s, eta: 3 days, 18:26:33 [2022/10/13 08:13:27] ppocr INFO: epoch: [21/600], global_step: 39480, lr: 0.000500, loss: 14.883772, score_loss: 0.082009, border_loss: 0.042347, direction_loss: 0.037831, ctc_loss: 2.946660, avg_reader_cost: 0.00010 s, avg_batch_cost: 0.20799 s, avg_samples: 4.0, ips: 19.23129 samples/s, eta: 3 days, 18:26:08 [2022/10/13 08:13:29] ppocr INFO: epoch: [21/600], global_step: 39490, lr: 0.000500, loss: 14.382166, score_loss: 0.086153, border_loss: 0.046786, direction_loss: 0.039923, ctc_loss: 2.842514, avg_reader_cost: 0.00010 s, avg_batch_cost: 0.20803 s, avg_samples: 4.0, ips: 19.22831 samples/s, eta: 3 days, 18:25:44 [2022/10/13 08:13:31] ppocr INFO: epoch: [21/600], global_step: 39500, lr: 0.000500, loss: 15.928978, score_loss: 0.090606, border_loss: 0.047891, direction_loss: 0.043698, ctc_loss: 3.152578, avg_reader_cost: 0.00011 s, avg_batch_cost: 0.20783 s, avg_samples: 4.0, ips: 19.24683 samples/s, eta: 3 days, 18:25:20 [2022/10/13 08:13:33] ppocr INFO: epoch: [21/600], global_step: 39510, lr: 0.000500, loss: 14.272430, score_loss: 0.082325, border_loss: 0.043174, direction_loss: 0.041341, ctc_loss: 2.822835, avg_reader_cost: 0.00010 s, avg_batch_cost: 0.20762 s, avg_samples: 4.0, ips: 19.26612 samples/s, eta: 3 days, 18:24:56 [2022/10/13 08:13:35] ppocr INFO: epoch: [21/600], global_step: 39520, lr: 0.000500, loss: 12.367882, score_loss: 0.075496, border_loss: 0.037698, direction_loss: 0.036265, ctc_loss: 2.437176, avg_reader_cost: 0.00010 s, avg_batch_cost: 0.20858 s, avg_samples: 4.0, ips: 19.17697 samples/s, eta: 3 days, 18:24:32 [2022/10/13 08:13:37] ppocr INFO: epoch: [21/600], global_step: 39530, lr: 0.000500, loss: 14.259925, score_loss: 0.085038, border_loss: 0.041370, direction_loss: 0.037612, ctc_loss: 2.820059, avg_reader_cost: 0.00011 s, avg_batch_cost: 0.20802 s, avg_samples: 4.0, ips: 19.22881 samples/s, eta: 3 days, 18:24:07 [2022/10/13 08:13:39] ppocr INFO: epoch: [21/600], global_step: 39540, lr: 0.000500, loss: 18.690460, score_loss: 0.085264, border_loss: 0.045676, direction_loss: 0.040817, ctc_loss: 3.706068, avg_reader_cost: 0.00010 s, avg_batch_cost: 0.20874 s, avg_samples: 4.0, ips: 19.16249 samples/s, eta: 3 days, 18:23:43 [2022/10/13 08:13:42] ppocr INFO: epoch: [21/600], global_step: 39550, lr: 0.000500, loss: 18.118183, score_loss: 0.082465, border_loss: 0.045710, direction_loss: 0.039703, ctc_loss: 3.580600, avg_reader_cost: 0.00010 s, avg_batch_cost: 0.20815 s, avg_samples: 4.0, ips: 19.21701 samples/s, eta: 3 days, 18:23:19 [2022/10/13 08:13:44] ppocr INFO: epoch: [21/600], global_step: 39560, lr: 0.000500, loss: 14.053546, score_loss: 0.081661, border_loss: 0.046653, direction_loss: 0.043053, ctc_loss: 2.779464, avg_reader_cost: 0.00010 s, avg_batch_cost: 0.20802 s, avg_samples: 4.0, ips: 19.22925 samples/s, eta: 3 days, 18:22:55 [2022/10/13 08:13:46] ppocr INFO: epoch: [21/600], global_step: 39570, lr: 0.000500, loss: 12.211974, score_loss: 0.080567, border_loss: 0.043780, direction_loss: 0.038907, ctc_loss: 2.410743, avg_reader_cost: 0.00010 s, avg_batch_cost: 0.20823 s, avg_samples: 4.0, ips: 19.20969 samples/s, eta: 3 days, 18:22:31 [2022/10/13 08:13:48] ppocr INFO: epoch: [21/600], global_step: 39580, lr: 0.000500, loss: 12.778858, score_loss: 0.083721, border_loss: 0.044429, direction_loss: 0.040825, ctc_loss: 2.519852, avg_reader_cost: 0.00011 s, avg_batch_cost: 0.20846 s, avg_samples: 4.0, ips: 19.18796 samples/s, eta: 3 days, 18:22:07 [2022/10/13 08:13:50] ppocr INFO: epoch: [21/600], global_step: 39590, lr: 0.000500, loss: 16.896507, score_loss: 0.086480, border_loss: 0.044519, direction_loss: 0.043292, ctc_loss: 3.342844, avg_reader_cost: 0.00010 s, avg_batch_cost: 0.20791 s, avg_samples: 4.0, ips: 19.23917 samples/s, eta: 3 days, 18:21:43 [2022/10/13 08:13:52] ppocr INFO: epoch: [21/600], global_step: 39600, lr: 0.000500, loss: 15.905052, score_loss: 0.081470, border_loss: 0.043193, direction_loss: 0.041853, ctc_loss: 3.145921, avg_reader_cost: 0.00010 s, avg_batch_cost: 0.20869 s, avg_samples: 4.0, ips: 19.16723 samples/s, eta: 3 days, 18:21:19 [2022/10/13 08:13:54] ppocr INFO: epoch: [21/600], global_step: 39610, lr: 0.000500, loss: 11.755800, score_loss: 0.085434, border_loss: 0.046411, direction_loss: 0.045020, ctc_loss: 2.310623, avg_reader_cost: 0.00010 s, avg_batch_cost: 0.20803 s, avg_samples: 4.0, ips: 19.22845 samples/s, eta: 3 days, 18:20:55 [2022/10/13 08:13:56] ppocr INFO: epoch: [21/600], global_step: 39620, lr: 0.000500, loss: 13.740167, score_loss: 0.085434, border_loss: 0.043384, direction_loss: 0.045551, ctc_loss: 2.711545, avg_reader_cost: 0.00010 s, avg_batch_cost: 0.20806 s, avg_samples: 4.0, ips: 19.22549 samples/s, eta: 3 days, 18:20:31 [2022/10/13 08:13:59] ppocr INFO: epoch: [21/600], global_step: 39630, lr: 0.000500, loss: 11.545247, score_loss: 0.078439, border_loss: 0.039450, direction_loss: 0.036063, ctc_loss: 2.280672, avg_reader_cost: 0.00010 s, avg_batch_cost: 0.20840 s, avg_samples: 4.0, ips: 19.19371 samples/s, eta: 3 days, 18:20:07 [2022/10/13 08:14:01] ppocr INFO: epoch: [21/600], global_step: 39640, lr: 0.000500, loss: 13.135178, score_loss: 0.081312, border_loss: 0.040138, direction_loss: 0.035810, ctc_loss: 2.589530, avg_reader_cost: 0.00011 s, avg_batch_cost: 0.20854 s, avg_samples: 4.0, ips: 19.18130 samples/s, eta: 3 days, 18:19:43 [2022/10/13 08:14:03] ppocr INFO: epoch: [21/600], global_step: 39650, lr: 0.000500, loss: 15.817850, score_loss: 0.086383, border_loss: 0.045921, direction_loss: 0.042403, ctc_loss: 3.130274, avg_reader_cost: 0.00014 s, avg_batch_cost: 0.21794 s, avg_samples: 4.0, ips: 18.35382 samples/s, eta: 3 days, 18:19:22 [2022/10/13 08:14:05] ppocr INFO: epoch: [21/600], global_step: 39660, lr: 0.000500, loss: 16.036316, score_loss: 0.088378, border_loss: 0.044603, direction_loss: 0.042403, ctc_loss: 3.173037, avg_reader_cost: 0.00016 s, avg_batch_cost: 0.22899 s, avg_samples: 4.0, ips: 17.46778 samples/s, eta: 3 days, 18:19:04 [2022/10/13 08:14:08] ppocr INFO: epoch: [21/600], global_step: 39670, lr: 0.000500, loss: 17.416771, score_loss: 0.082245, border_loss: 0.042485, direction_loss: 0.041155, ctc_loss: 3.446843, avg_reader_cost: 0.00017 s, avg_batch_cost: 0.22871 s, avg_samples: 4.0, ips: 17.48956 samples/s, eta: 3 days, 18:18:46 [2022/10/13 08:14:10] ppocr INFO: epoch: [21/600], global_step: 39680, lr: 0.000500, loss: 14.509926, score_loss: 0.081122, border_loss: 0.041379, direction_loss: 0.041736, ctc_loss: 2.866924, avg_reader_cost: 0.00015 s, avg_batch_cost: 0.23110 s, avg_samples: 4.0, ips: 17.30862 samples/s, eta: 3 days, 18:18:29 [2022/10/13 08:14:13] ppocr INFO: epoch: [21/600], global_step: 39690, lr: 0.000500, loss: 11.888916, score_loss: 0.084780, border_loss: 0.044396, direction_loss: 0.041077, ctc_loss: 2.344761, avg_reader_cost: 0.00018 s, avg_batch_cost: 0.26457 s, avg_samples: 4.0, ips: 15.11893 samples/s, eta: 3 days, 18:18:21 [2022/10/13 08:14:15] ppocr INFO: epoch: [21/600], global_step: 39700, lr: 0.000500, loss: 15.392438, score_loss: 0.085457, border_loss: 0.044396, direction_loss: 0.040781, ctc_loss: 3.044455, avg_reader_cost: 0.00015 s, avg_batch_cost: 0.26015 s, avg_samples: 4.0, ips: 15.37592 samples/s, eta: 3 days, 18:18:12 [2022/10/13 08:14:18] ppocr INFO: epoch: [21/600], global_step: 39710, lr: 0.000500, loss: 16.753483, score_loss: 0.086103, border_loss: 0.045940, direction_loss: 0.039138, ctc_loss: 3.312874, avg_reader_cost: 0.00016 s, avg_batch_cost: 0.26454 s, avg_samples: 4.0, ips: 15.12040 samples/s, eta: 3 days, 18:18:05 [2022/10/13 08:14:21] ppocr INFO: epoch: [21/600], global_step: 39720, lr: 0.000500, loss: 15.308136, score_loss: 0.084307, border_loss: 0.043013, direction_loss: 0.038811, ctc_loss: 3.023912, avg_reader_cost: 0.00018 s, avg_batch_cost: 0.26263 s, avg_samples: 4.0, ips: 15.23080 samples/s, eta: 3 days, 18:17:57 [2022/10/13 08:14:23] ppocr INFO: epoch: [21/600], global_step: 39730, lr: 0.000500, loss: 13.452417, score_loss: 0.082696, border_loss: 0.040619, direction_loss: 0.038531, ctc_loss: 2.658642, avg_reader_cost: 0.00019 s, avg_batch_cost: 0.26258 s, avg_samples: 4.0, ips: 15.23374 samples/s, eta: 3 days, 18:17:49 [2022/10/13 08:14:25] ppocr INFO: epoch: [21/600], global_step: 39740, lr: 0.000500, loss: 12.475712, score_loss: 0.083187, border_loss: 0.041558, direction_loss: 0.039931, ctc_loss: 2.463814, avg_reader_cost: 0.00014 s, avg_batch_cost: 0.20954 s, avg_samples: 4.0, ips: 19.08977 samples/s, eta: 3 days, 18:17:25 [2022/10/13 08:14:28] ppocr INFO: epoch: [21/600], global_step: 39750, lr: 0.000500, loss: 10.151573, score_loss: 0.082312, border_loss: 0.042810, direction_loss: 0.039848, ctc_loss: 1.993367, avg_reader_cost: 0.00011 s, avg_batch_cost: 0.20822 s, avg_samples: 4.0, ips: 19.20999 samples/s, eta: 3 days, 18:17:01 [2022/10/13 08:14:30] ppocr INFO: epoch: [21/600], global_step: 39760, lr: 0.000500, loss: 13.987549, score_loss: 0.086700, border_loss: 0.043557, direction_loss: 0.040781, ctc_loss: 2.767547, avg_reader_cost: 0.00012 s, avg_batch_cost: 0.20869 s, avg_samples: 4.0, ips: 19.16755 samples/s, eta: 3 days, 18:16:38 [2022/10/13 08:14:32] ppocr INFO: epoch: [21/600], global_step: 39770, lr: 0.000500, loss: 15.021500, score_loss: 0.090701, border_loss: 0.050564, direction_loss: 0.047886, ctc_loss: 2.962197, avg_reader_cost: 0.00011 s, avg_batch_cost: 0.20842 s, avg_samples: 4.0, ips: 19.19175 samples/s, eta: 3 days, 18:16:14 [2022/10/13 08:14:34] ppocr INFO: epoch: [21/600], global_step: 39780, lr: 0.000500, loss: 12.595399, score_loss: 0.087435, border_loss: 0.050079, direction_loss: 0.045493, ctc_loss: 2.489696, avg_reader_cost: 0.00011 s, avg_batch_cost: 0.20830 s, avg_samples: 4.0, ips: 19.20296 samples/s, eta: 3 days, 18:15:50 [2022/10/13 08:14:36] ppocr INFO: epoch: [21/600], global_step: 39790, lr: 0.000500, loss: 12.420128, score_loss: 0.087882, border_loss: 0.046493, direction_loss: 0.041497, ctc_loss: 2.454984, avg_reader_cost: 0.00011 s, avg_batch_cost: 0.20901 s, avg_samples: 4.0, ips: 19.13805 samples/s, eta: 3 days, 18:15:26 [2022/10/13 08:14:38] ppocr INFO: epoch: [21/600], global_step: 39800, lr: 0.000500, loss: 12.952734, score_loss: 0.086056, border_loss: 0.042755, direction_loss: 0.040319, ctc_loss: 2.554852, avg_reader_cost: 0.00011 s, avg_batch_cost: 0.20840 s, avg_samples: 4.0, ips: 19.19420 samples/s, eta: 3 days, 18:15:02 [2022/10/13 08:14:40] ppocr INFO: epoch: [21/600], global_step: 39810, lr: 0.000500, loss: 14.896111, score_loss: 0.085481, border_loss: 0.043563, direction_loss: 0.040067, ctc_loss: 2.950872, avg_reader_cost: 0.00011 s, avg_batch_cost: 0.20889 s, avg_samples: 4.0, ips: 19.14859 samples/s, eta: 3 days, 18:14:39 [2022/10/13 08:14:42] ppocr INFO: epoch: [21/600], global_step: 39820, lr: 0.000500, loss: 16.545589, score_loss: 0.084761, border_loss: 0.046921, direction_loss: 0.046003, ctc_loss: 3.275897, avg_reader_cost: 0.00013 s, avg_batch_cost: 0.20868 s, avg_samples: 4.0, ips: 19.16821 samples/s, eta: 3 days, 18:14:15 [2022/10/13 08:14:45] ppocr INFO: epoch: [21/600], global_step: 39830, lr: 0.000500, loss: 12.593184, score_loss: 0.081044, border_loss: 0.043594, direction_loss: 0.046328, ctc_loss: 2.482746, avg_reader_cost: 0.00011 s, avg_batch_cost: 0.20835 s, avg_samples: 4.0, ips: 19.19851 samples/s, eta: 3 days, 18:13:51 [2022/10/13 08:14:47] ppocr INFO: epoch: [21/600], global_step: 39840, lr: 0.000500, loss: 12.095426, score_loss: 0.083726, border_loss: 0.041615, direction_loss: 0.038355, ctc_loss: 2.388390, avg_reader_cost: 0.00011 s, avg_batch_cost: 0.20870 s, avg_samples: 4.0, ips: 19.16640 samples/s, eta: 3 days, 18:13:28 [2022/10/13 08:14:49] ppocr INFO: epoch: [21/600], global_step: 39850, lr: 0.000500, loss: 12.987989, score_loss: 0.085774, border_loss: 0.043293, direction_loss: 0.038985, ctc_loss: 2.561947, avg_reader_cost: 0.00011 s, avg_batch_cost: 0.20843 s, avg_samples: 4.0, ips: 19.19103 samples/s, eta: 3 days, 18:13:04 [2022/10/13 08:14:51] ppocr INFO: epoch: [21/600], global_step: 39860, lr: 0.000500, loss: 12.736959, score_loss: 0.088678, border_loss: 0.044359, direction_loss: 0.041414, ctc_loss: 2.512642, avg_reader_cost: 0.00012 s, avg_batch_cost: 0.20846 s, avg_samples: 4.0, ips: 19.18863 samples/s, eta: 3 days, 18:12:40 [2022/10/13 08:14:53] ppocr INFO: epoch: [21/600], global_step: 39870, lr: 0.000500, loss: 12.868677, score_loss: 0.089206, border_loss: 0.044987, direction_loss: 0.043499, ctc_loss: 2.539112, avg_reader_cost: 0.00011 s, avg_batch_cost: 0.20854 s, avg_samples: 4.0, ips: 19.18071 samples/s, eta: 3 days, 18:12:17 [2022/10/13 08:14:55] ppocr INFO: epoch: [21/600], global_step: 39880, lr: 0.000500, loss: 12.596214, score_loss: 0.080797, border_loss: 0.041196, direction_loss: 0.041459, ctc_loss: 2.487576, avg_reader_cost: 0.00012 s, avg_batch_cost: 0.20844 s, avg_samples: 4.0, ips: 19.18996 samples/s, eta: 3 days, 18:11:53 [2022/10/13 08:14:57] ppocr INFO: epoch: [21/600], global_step: 39890, lr: 0.000500, loss: 14.421125, score_loss: 0.082208, border_loss: 0.041546, direction_loss: 0.041352, ctc_loss: 2.848225, avg_reader_cost: 0.00012 s, avg_batch_cost: 0.20839 s, avg_samples: 4.0, ips: 19.19515 samples/s, eta: 3 days, 18:11:29 [2022/10/13 08:14:59] ppocr INFO: epoch: [21/600], global_step: 39900, lr: 0.000500, loss: 17.430897, score_loss: 0.087352, border_loss: 0.046538, direction_loss: 0.047067, ctc_loss: 3.450174, avg_reader_cost: 0.00011 s, avg_batch_cost: 0.20824 s, avg_samples: 4.0, ips: 19.20886 samples/s, eta: 3 days, 18:11:05 [2022/10/13 08:15:02] ppocr INFO: epoch: [21/600], global_step: 39910, lr: 0.000500, loss: 20.317698, score_loss: 0.090101, border_loss: 0.049045, direction_loss: 0.047578, ctc_loss: 4.025415, avg_reader_cost: 0.00011 s, avg_batch_cost: 0.20816 s, avg_samples: 4.0, ips: 19.21575 samples/s, eta: 3 days, 18:10:42 [2022/10/13 08:15:04] ppocr INFO: epoch: [21/600], global_step: 39920, lr: 0.000500, loss: 19.807043, score_loss: 0.086784, border_loss: 0.045171, direction_loss: 0.037915, ctc_loss: 3.921174, avg_reader_cost: 0.00011 s, avg_batch_cost: 0.20875 s, avg_samples: 4.0, ips: 19.16203 samples/s, eta: 3 days, 18:10:18 [2022/10/13 08:15:06] ppocr INFO: epoch: [21/600], global_step: 39930, lr: 0.000500, loss: 15.155674, score_loss: 0.083684, border_loss: 0.041499, direction_loss: 0.038533, ctc_loss: 2.991148, avg_reader_cost: 0.00011 s, avg_batch_cost: 0.20857 s, avg_samples: 4.0, ips: 19.17860 samples/s, eta: 3 days, 18:09:54 [2022/10/13 08:15:08] ppocr INFO: epoch: [21/600], global_step: 39940, lr: 0.000500, loss: 14.440382, score_loss: 0.082268, border_loss: 0.042695, direction_loss: 0.043694, ctc_loss: 2.856018, avg_reader_cost: 0.00012 s, avg_batch_cost: 0.20829 s, avg_samples: 4.0, ips: 19.20383 samples/s, eta: 3 days, 18:09:31 [2022/10/13 08:15:10] ppocr INFO: epoch: [21/600], global_step: 39950, lr: 0.000500, loss: 17.541693, score_loss: 0.085058, border_loss: 0.044782, direction_loss: 0.040903, ctc_loss: 3.464224, avg_reader_cost: 0.00012 s, avg_batch_cost: 0.20859 s, avg_samples: 4.0, ips: 19.17592 samples/s, eta: 3 days, 18:09:07 [2022/10/13 08:15:12] ppocr INFO: epoch: [21/600], global_step: 39960, lr: 0.000500, loss: 16.219200, score_loss: 0.089630, border_loss: 0.046373, direction_loss: 0.040587, ctc_loss: 3.199691, avg_reader_cost: 0.00012 s, avg_batch_cost: 0.20848 s, avg_samples: 4.0, ips: 19.18630 samples/s, eta: 3 days, 18:08:44 [2022/10/13 08:15:14] ppocr INFO: epoch: [21/600], global_step: 39970, lr: 0.000500, loss: 15.549601, score_loss: 0.088411, border_loss: 0.045054, direction_loss: 0.041444, ctc_loss: 3.067267, avg_reader_cost: 0.00011 s, avg_batch_cost: 0.20854 s, avg_samples: 4.0, ips: 19.18129 samples/s, eta: 3 days, 18:08:20 [2022/10/13 08:15:16] ppocr INFO: epoch: [21/600], global_step: 39980, lr: 0.000500, loss: 15.399892, score_loss: 0.084734, border_loss: 0.040278, direction_loss: 0.040536, ctc_loss: 3.043568, avg_reader_cost: 0.00007 s, avg_batch_cost: 0.20765 s, avg_samples: 4.0, ips: 19.26294 samples/s, eta: 3 days, 18:07:56 [2022/10/13 08:15:19] ppocr INFO: epoch: [21/600], global_step: 39990, lr: 0.000500, loss: 10.905333, score_loss: 0.085139, border_loss: 0.042850, direction_loss: 0.038516, ctc_loss: 2.145095, avg_reader_cost: 0.00007 s, avg_batch_cost: 0.20760 s, avg_samples: 4.0, ips: 19.26790 samples/s, eta: 3 days, 18:07:32 [2022/10/13 08:15:21] ppocr INFO: epoch: [21/600], global_step: 40000, lr: 0.000500, loss: 10.912173, score_loss: 0.076996, border_loss: 0.039444, direction_loss: 0.034958, ctc_loss: 2.147245, avg_reader_cost: 0.00006 s, avg_batch_cost: 0.20764 s, avg_samples: 4.0, ips: 19.26412 samples/s, eta: 3 days, 18:07:09 [2022/10/13 08:18:44] ppocr INFO: cur metric, total_num_gt: 5047, total_num_det: 17, global_accumulative_recall: 0, hit_str_count: 0, recall: 0.0, precision: 0.0, f_score: 0, seqerr: 1, recall_e2e: 0.0, precision_e2e: 0.0, f_score_e2e: 0, fps: 27.355039048819574 [2022/10/13 08:18:45] ppocr INFO: save best model is to ./output/pgnet_r50_vd_totaltext/best_accuracy [2022/10/13 08:18:45] ppocr INFO: best metric, f_score_e2e: 0, is_float16: False, total_num_gt: 5047, total_num_det: 17, global_accumulative_recall: 0, hit_str_count: 0, recall: 0.0, precision: 0.0, f_score: 0, seqerr: 1, recall_e2e: 0.0, precision_e2e: 0.0, fps: 27.355039048819574, best_epoch: 21, start_epoch: 2 [2022/10/13 08:18:45] ppocr INFO: save model in ./output/pgnet_r50_vd_totaltext/latest
其中配置文件中使用的字符集字典是自己根据印章上的文字取下来组成的,配置文件如下: `Global: use_gpu: True epoch_num: 600 log_smooth_window: 20 print_batch_step: 10 save_model_dir: ./output/pgnet_r50_vd_totaltext/ save_epoch_step: 10
evaluation is run every 0 iterationss after the 1000th iterationeval_batch_step: [ 0, 1000 ] cal_metric_during_train: False pretrained_model: checkpoints: ./output/pgnet_r50_vd_totaltext/best_accuracy save_inference_dir: ./inference_img/ use_visualdl: False infer_img: valid_set: totaltext # two mode: totaltext valid curved words, partvgg valid non-curved words save_res_path: ./output/pgnet_r50_vd_totaltext/predicts_pgnet.txt character_dict_path: ppocr/utils/ch_ppor_keys.txt character_type: CN max_text_length: 50 # the max length in seq max_text_nums: 30 # the max seq nums in a pic tcl_len: 64
Architecture: model_type: e2e algorithm: PGNet Transform: Backbone: name: ResNet layers: 50 Neck: name: PGFPN Head: name: PGHead
Loss: name: PGLoss tcl_bs: 64 max_text_length: 50 # the same as Global: max_text_length max_text_nums: 30 # the same as Global:max_text_nums pad_num: 2202 # the length of dict for pad 字符集字典长度
Optimizer: name: Adam beta1: 0.9 beta2: 0.999 lr: learning_rate: 0.0005 regularizer: name: 'L2' factor: 0
PostProcess: name: PGPostProcess score_thresh: 0.5 mode: fast # fast or slow two ways
Metric: name: E2EMetric mode: A # two ways for eval, A: label from txt, B: label from gt_mat
gt_mat_dir: ./train_data/total_text/gt # the dir of gt_matgt_mat_dir: ./dataset/test/test.txt character_dict_path: ppocr/utils/ch_ppor_keys.txt main_indicator: f_score_e2e
Train: dataset: name: PGDataSet data_dir: ./dataset/train/rgb/ label_file_list: [./dataset/train/train.txt] ratio_list: [1.0] transforms:
- DecodeImage: # load image img_mode: BGR channel_first: False
- E2ELabelEncodeTrain:
- PGProcessTrain: batch_size: 4 # same as loader: batch_size_per_card min_crop_size: 24 min_text_size: 4 max_text_size: 512
- KeepKeys: keep_keys: [ 'images', 'tcl_maps', 'tcl_label_maps', 'border_maps','direction_maps', 'training_masks', 'label_list', 'pos_list', 'pos_mask' ] # dataloader will return list in this order loader: shuffle: True drop_last: True batch_size_per_card: 4 num_workers: 16
Eval: dataset: name: PGDataSet data_dir: ./dataset/test/rgb/ label_file_list: [./dataset/test/test.txt] transforms:
- DecodeImage: # load image img_mode: BGR channel_first: False
- E2ELabelEncodeTest:
- E2EResizeForTest: max_side_len: 768
- NormalizeImage: scale: 1./255. mean: [ 0.485, 0.456, 0.406 ] std: [ 0.229, 0.224, 0.225 ] order: 'hwc'
- ToCHWImage:
- KeepKeys: keep_keys: [ 'image', 'shape', 'polys', 'texts', 'ignore_tags', 'img_id'] loader: shuffle: False drop_last: False batch_size_per_card: 1 # must be 1 num_workers: 0 ` 关键性的PaddleOCR/ppocr/modeling/heads/e2e_pg_head.py /中181行的out_channels=设置成字符集字典数目+1 ppocr/losses/e2e_pg_loss.py中93行改为字符集字典数目+1,94和97行改为字符集字典数目 配置文件中Loss的pad_num已改成字符集字典长度。 但是,训练了21个epochs,验证的指标还全部是0,请问这是出了什么错误吗?