8
-PS:自制数据集链接：文档：公开数据集 链接：http://note.youdao.com/noteshare?id=754b970aa21174543f97cddec88c2a34
- 系统环境/System Environment：UBUNTU 20.04
- 版本号/Version：Paddle： PaddleOCR： 问题相关组件/Related components：PGNET
- 运行指令/Command
Code：python3 tools/train.py -c configs/e2e/e2e_r50_vd_pg.yml -o Global.pretrained_model=./pretrain_models/train_step1/best_accuracy Global。 load_static_weights=False - 完整报错/Complete Error Message：
- `(paddle_ocr_det) jeck@jeck-LEGION-REN7000K-26IOB:~/Public/PaddleOCR$ python3 tools/train.py -c configs/e2e/e2e_r50_vd_pg.yml -o Global.pretrained_model=./pretrain_models/train_step1/best_accuracy Global.load_static_weights=False
[2022/02/18 17:24:51] root INFO: Architecture :
[2022/02/18 17:24:51] root INFO: Backbone :
[2022/02/18 17:24:51] root INFO: layers : 50
[2022/02/18 17:24:51] root INFO: name : ResNet
[2022/02/18 17:24:51] root INFO: Head :
[2022/02/18 17:24:51] root INFO: name : PGHead
[2022/02/18 17:24:51] root INFO: Neck :
[2022/02/18 17:24:51] root INFO: name : PGFPN
[2022/02/18 17:24:51] root INFO: Transform : None
[2022/02/18 17:24:51] root INFO: algorithm : PGNet
[2022/02/18 17:24:51] root INFO: model_type : e2e
[2022/02/18 17:24:51] root INFO: Eval :
[2022/02/18 17:24:51] root INFO: dataset :
[2022/02/18 17:24:51] root INFO: data_dir : ./train_data/total_text/test
[2022/02/18 17:24:51] root INFO: label_file_list : ['./train_data/total_text/test/test.txt']
[2022/02/18 17:24:51] root INFO: name : PGDataSet
[2022/02/18 17:24:51] root INFO: transforms :
[2022/02/18 17:24:51] root INFO: DecodeImage :
[2022/02/18 17:24:51] root INFO: channel_first : False
[2022/02/18 17:24:51] root INFO: img_mode : BGR
[2022/02/18 17:24:51] root INFO: E2ELabelEncodeTest : None
[2022/02/18 17:24:51] root INFO: E2EResizeForTest :
[2022/02/18 17:24:51] root INFO: max_side_len : 768
[2022/02/18 17:24:51] root INFO: NormalizeImage :
[2022/02/18 17:24:51] root INFO: mean : [0.485, 0.456, 0.406]
[2022/02/18 17:24:51] root INFO: order : hwc
[2022/02/18 17:24:51] root INFO: scale : 1./255.
[2022/02/18 17:24:51] root INFO: std : [0.229, 0.224, 0.225]
[2022/02/18 17:24:51] root INFO: ToCHWImage : None
[2022/02/18 17:24:51] root INFO: KeepKeys :
[2022/02/18 17:24:51] root INFO: keep_keys : ['image', 'shape', 'polys', 'texts', 'ignore_tags', 'img_id']
[2022/02/18 17:24:51] root INFO: loader :
[2022/02/18 17:24:51] root INFO: batch_size_per_card : 1
[2022/02/18 17:24:51] root INFO: drop_last : False
[2022/02/18 17:24:51] root INFO: num_workers : 2
[2022/02/18 17:24:51] root INFO: shuffle : False
[2022/02/18 17:24:51] root INFO: Global :
[2022/02/18 17:24:51] root INFO: cal_metric_during_train : False
[2022/02/18 17:24:51] root INFO: character_dict_path : ppocr/utils/ic15_dict.txt
[2022/02/18 17:24:51] root INFO: character_type : EN
[2022/02/18 17:24:51] root INFO: checkpoints : None
[2022/02/18 17:24:51] root INFO: debug : False
[2022/02/18 17:24:51] root INFO: distributed : False
[2022/02/18 17:24:51] root INFO: epoch_num : 1500
[2022/02/18 17:24:51] root INFO: eval_batch_step : [0, 1000]
[2022/02/18 17:24:51] root INFO: infer_img : None
[2022/02/18 17:24:51] root INFO: load_static_weights : False
[2022/02/18 17:24:51] root INFO: log_smooth_window : 20
[2022/02/18 17:24:51] root INFO: max_text_length : 50
[2022/02/18 17:24:51] root INFO: max_text_nums : 30
[2022/02/18 17:24:51] root INFO: pretrained_model : ./pretrain_models/train_step1/best_accuracy
[2022/02/18 17:24:51] root INFO: print_batch_step : 10
[2022/02/18 17:24:51] root INFO: save_epoch_step : 10
[2022/02/18 17:24:51] root INFO: save_inference_dir : None
[2022/02/18 17:24:51] root INFO: save_model_dir : ./output/pgnet_r50_vd_totaltext_0216/
[2022/02/18 17:24:51] root INFO: save_res_path : ./output/pgnet_r50_vd_totaltext_0216/predicts_pgnet.txt
[2022/02/18 17:24:51] root INFO: tcl_len : 64
[2022/02/18 17:24:51] root INFO: use_gpu : True
[2022/02/18 17:24:51] root INFO: use_visualdl : False
[2022/02/18 17:24:51] root INFO: valid_set : totaltext
[2022/02/18 17:24:51] root INFO: Loss :
[2022/02/18 17:24:51] root INFO: max_text_length : 50
[2022/02/18 17:24:51] root INFO: max_text_nums : 30
[2022/02/18 17:24:51] root INFO: name : PGLoss
[2022/02/18 17:24:51] root INFO: pad_num : 36
[2022/02/18 17:24:51] root INFO: tcl_bs : 64
[2022/02/18 17:24:51] root INFO: Metric :
[2022/02/18 17:24:51] root INFO: character_dict_path : ppocr/utils/ic15_dict.txt
[2022/02/18 17:24:51] root INFO: gt_mat_dir : ./train_data/total_text/gt
[2022/02/18 17:24:51] root INFO: main_indicator : f_score_e2e
[2022/02/18 17:24:51] root INFO: mode : A
[2022/02/18 17:24:51] root INFO: name : E2EMetric
[2022/02/18 17:24:51] root INFO: Optimizer :
[2022/02/18 17:24:51] root INFO: beta1 : 0.9
[2022/02/18 17:24:51] root INFO: beta2 : 0.999
[2022/02/18 17:24:51] root INFO: lr :
[2022/02/18 17:24:51] root INFO: learning_rate : 0.001
[2022/02/18 17:24:51] root INFO: name : Adam
[2022/02/18 17:24:51] root INFO: regularizer :
[2022/02/18 17:24:51] root INFO: factor : 0
[2022/02/18 17:24:51] root INFO: name : L2
[2022/02/18 17:24:51] root INFO: PostProcess :
[2022/02/18 17:24:51] root INFO: mode : fast
[2022/02/18 17:24:51] root INFO: name : PGPostProcess
[2022/02/18 17:24:51] root INFO: score_thresh : 0.5
[2022/02/18 17:24:51] root INFO: Train :
[2022/02/18 17:24:51] root INFO: dataset :
[2022/02/18 17:24:51] root INFO: data_dir : ./train_data/total_text/train
[2022/02/18 17:24:51] root INFO: label_file_list : ['./train_data/total_text/train/train.txt']
[2022/02/18 17:24:51] root INFO: name : PGDataSet
[2022/02/18 17:24:51] root INFO: ratio_list : [1.0]
[2022/02/18 17:24:51] root INFO: transforms :
[2022/02/18 17:24:51] root INFO: DecodeImage :
[2022/02/18 17:24:51] root INFO: channel_first : False
[2022/02/18 17:24:51] root INFO: img_mode : BGR
[2022/02/18 17:24:51] root INFO: E2ELabelEncodeTrain : None
[2022/02/18 17:24:51] root INFO: PGProcessTrain :
[2022/02/18 17:24:51] root INFO: batch_size : 14
[2022/02/18 17:24:51] root INFO: max_text_size : 512
[2022/02/18 17:24:51] root INFO: min_crop_size : 24
[2022/02/18 17:24:51] root INFO: min_text_size : 4
[2022/02/18 17:24:51] root INFO: KeepKeys :
[2022/02/18 17:24:51] root INFO: keep_keys : ['images', 'tcl_maps', 'tcl_label_maps', 'border_maps', 'direction_maps', 'training_masks', 'label_list', 'pos_list', 'pos_mask']
[2022/02/18 17:24:51] root INFO: loader :
[2022/02/18 17:24:51] root INFO: batch_size_per_card : 14
[2022/02/18 17:24:51] root INFO: drop_last : True
[2022/02/18 17:24:51] root INFO: num_workers : 16
[2022/02/18 17:24:51] root INFO: shuffle : True
[2022/02/18 17:24:51] root INFO: train with paddle 2.2.1 and device CUDAPlace(0)
[2022/02/18 17:24:51] root INFO: Initialize indexs of datasets:['./train_data/total_text/train/train.txt']
[2022/02/18 17:24:51] root INFO: Initialize indexs of datasets:['./train_data/total_text/test/test.txt']
W0218 17:24:51.900465 4857 device_context.cc:447] Please NOTE: device: 0, GPU Compute Capability: 8.6, Driver API Version: 11.4, Runtime API Version: 11.2
W0218 17:24:51.902981 4857 device_context.cc:465] device: 0, cuDNN Version: 8.1.
[2022/02/18 17:24:54] root INFO: loaded pretrained_model successful from ./pretrain_models/train_step1/best_accuracy.pdparams
[2022/02/18 17:24:54] root INFO: train dataloader has 16 iters
[2022/02/18 17:24:54] root INFO: valid dataloader has 91 iters
[2022/02/18 17:24:54] root INFO: During the training process, after the 0th iteration, an evaluation is run every 1000 iterations
[2022/02/18 17:24:54] root INFO: Initialize indexs of datasets:['./train_data/total_text/train/train.txt']
[2022/02/18 17:24:54] root ERROR: When parsing line 181, error happened with msg: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.
[2022/02/18 17:24:54] root ERROR: When parsing line 107, error happened with msg: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.
[2022/02/18 17:24:54] root ERROR: When parsing line 200, error happened with msg: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (3,) + inhomogeneous part.
[2022/02/18 17:24:54] root ERROR: When parsing line 102, error happened with msg: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.
[2022/02/18 17:24:54] root ERROR: When parsing line 193, error happened with msg: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.
[2022/02/18 17:24:54] root ERROR: When parsing line 74, error happened with msg: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.
[2022/02/18 17:24:54] root ERROR: When parsing line 108, error happened with msg: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.
[2022/02/18 17:24:54] root ERROR: When parsing line 209, error happened with msg: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.
[2022/02/18 17:24:54] root ERROR: When parsing line 12, error happened with msg: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.
[2022/02/18 17:24:54] root ERROR: When parsing line 102, error happened with msg: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.
[2022/02/18 17:24:54] root ERROR: When parsing line 43, error happened with msg: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.
[2022/02/18 17:24:55] root ERROR: When parsing line 229, error happened with msg: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.
[2022/02/18 17:24:55] root ERROR: When parsing line 222, error happened with msg: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.
[2022/02/18 17:24:55] root ERROR: When parsing line 222, error happened with msg: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.
[2022/02/18 17:24:55] root ERROR: When parsing line 185, error happened with msg: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.
[2022/02/18 17:24:55] root ERROR: When parsing line 171, error happened with msg: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.
[2022/02/18 17:24:55] root ERROR: When parsing line 74, error happened with msg: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.
[2022/02/18 17:24:55] root ERROR: When parsing line 12, error happened with msg: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.
[2022/02/18 17:24:55] root ERROR: When parsing line 102, error happened with msg: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.
[2022/02/18 17:24:55] root ERROR: When parsing line 192, error happened with msg: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (4,) + inhomogeneous part.
[2022/02/18 17:24:55] root ERROR: When parsing line 229, error happened with msg: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.
[2022/02/18 17:24:55] root ERROR: When parsing line 157, error happened with msg: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.
[2022/02/18 17:24:55] root ERROR: When parsing line 193, error happened with msg: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.
[2022/02/18 17:24:55] root ERROR: When parsing line 115, error happened with msg: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.
[2022/02/18 17:24:55] root ERROR: When parsing line 30, error happened with msg: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.
[2022/02/18 17:24:55] root ERROR: When parsing line 185, error happened with msg: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.
[2022/02/18 17:24:55] root ERROR: When parsing line 111, error happened with msg: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.
[2022/02/18 17:24:55] root ERROR: When parsing line 181, error happened with msg: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.
[2022/02/18 17:24:55] root ERROR: When parsing line 43, error happened with msg: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.
[2022/02/18 17:24:55] root ERROR: When parsing line 73, error happened with msg: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.
[2022/02/18 17:24:55] root ERROR: When parsing line 74, error happened with msg: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.
[2022/02/18 17:24:55] root ERROR: When parsing line 222, error happened with msg: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.
[2022/02/18 17:24:55] root ERROR: When parsing line 130, error happened with msg: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (3,) + inhomogeneous part.
[2022/02/18 17:24:55] root ERROR: When parsing line 200, error happened with msg: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (3,) + inhomogeneous part.
[2022/02/18 17:24:55] root ERROR: When parsing line 229, error happened with msg: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.
[2022/02/18 17:24:55] root ERROR: When parsing line 209, error happened with msg: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.
[2022/02/18 17:24:55] root ERROR: When parsing line 73, error happened with msg: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.
[2022/02/18 17:24:55] root ERROR: When parsing line 95, error happened with msg: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.
[2022/02/18 17:24:55] root ERROR: When parsing line 92, error happened with msg: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.
[2022/02/18 17:24:55] root ERROR: When parsing line 73, error happened with msg: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.
[2022/02/18 17:24:55] root ERROR: When parsing line 43, error happened with msg: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.
[2022/02/18 17:24:55] root ERROR: When parsing line 87, error happened with msg: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.
[2022/02/18 17:24:55] root ERROR: When parsing line 156, error happened with msg: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.
[2022/02/18 17:24:55] root ERROR: When parsing line 209, error happened with msg: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.
[2022/02/18 17:24:55] root ERROR: When parsing line 185, error happened with msg: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.
[2022/02/18 17:24:55] root ERROR: When parsing line 30, error happened with msg: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.
[2022/02/18 17:24:55] root ERROR: When parsing line 102, error happened with msg: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.
[2022/02/18 17:24:55] root ERROR: When parsing line 137, error happened with msg: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.
[2022/02/18 17:24:56] root ERROR: When parsing line 95, error happened with msg: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.
[2022/02/18 17:24:56] root ERROR: When parsing line 193, error happened with msg: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.
[2022/02/18 17:24:56] root ERROR: When parsing line 30, error happened with msg: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.
[2022/02/18 17:24:56] root ERROR: When parsing line 73, error happened with msg: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.
[2022/02/18 17:24:56] root ERROR: When parsing line 161, error happened with msg: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.
[2022/02/18 17:24:56] root ERROR: When parsing line 30, error happened with msg: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.
[2022/02/18 17:24:56] root ERROR: When parsing line 102, error happened with msg: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.
[2022/02/18 17:24:56] root ERROR: When parsing line 137, error happened with msg: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.
[2022/02/18 17:24:56] root ERROR: When parsing line 185, error happened with msg: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.
[2022/02/18 17:24:56] root ERROR: When parsing line 106, error happened with msg: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.
[2022/02/18 17:24:56] root ERROR: When parsing line 193, error happened with msg: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.
[2022/02/18 17:24:56] root ERROR: When parsing line 171, error happened with msg: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.
[2022/02/18 17:24:56] root ERROR: When parsing line 130, error happened with msg: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (3,) + inhomogeneous part.
[2022/02/18 17:24:56] root ERROR: When parsing line 92, error happened with msg: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.
[2022/02/18 17:24:56] root ERROR: When parsing line 192, error happened with msg: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (4,) + inhomogeneous part.
[2022/02/18 17:24:56] root ERROR: When parsing line 171, error happened with msg: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.
[2022/02/18 17:24:56] root ERROR: When parsing line 129, error happened with msg: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (4,) + inhomogeneous part.
[2022/02/18 17:24:56] root ERROR: When parsing line 157, error happened with msg: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.
[2022/02/18 17:24:56] root ERROR: When parsing line 156, error happened with msg: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.
[2022/02/18 17:24:56] root ERROR: When parsing line 87, error happened with msg: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.
[2022/02/18 17:24:56] root ERROR: When parsing line 161, error happened with msg: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.
[2022/02/18 17:24:56] root ERROR: When parsing line 145, error happened with msg: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.
[2022/02/18 17:24:56] root ERROR: When parsing line 108, error happened with msg: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.
[2022/02/18 17:24:56] root ERROR: When parsing line 106, error happened with msg: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.
[2022/02/18 17:24:56] root ERROR: When parsing line 129, error happened with msg: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (4,) + inhomogeneous part.
[2022/02/18 17:24:56] root ERROR: When parsing line 156, error happened with msg: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.
[2022/02/18 17:24:56] root ERROR: When parsing line 137, error happened with msg: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.
[2022/02/18 17:24:56] root ERROR: When parsing line 222, error happened with msg: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.
[2022/02/18 17:24:56] root ERROR: When parsing line 229, error happened with msg: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.
[2022/02/18 17:24:56] root ERROR: When parsing line 156, error happened with msg: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.
[2022/02/18 17:24:56] root ERROR: When parsing line 222, error happened with msg: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.
[2022/02/18 17:24:56] root ERROR: When parsing line 157, error happened with msg: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.
[2022/02/18 17:24:57] root ERROR: When parsing line 12, error happened with msg: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.
[2022/02/18 17:24:57] root ERROR: When parsing line 222, error happened with msg: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.
[2022/02/18 17:24:57] root ERROR: When parsing line 185, error happened with msg: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.
[2022/02/18 17:24:57] root ERROR: When parsing line 229, error happened with msg: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.
[2022/02/18 17:24:57] root ERROR: When parsing line 43, error happened with msg: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.
[2022/02/18 17:24:57] root ERROR: When parsing line 185, error happened with msg: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.
[2022/02/18 17:24:57] root ERROR: When parsing line 161, error happened with msg: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.
Traceback (most recent call last):
File "tools/train.py", line 131, in
main(config, device, logger, vdl_writer) File "tools/train.py", line 106, in main program.train(config, train_dataloader, valid_dataloader, device, model, File "/home/jeck/Public/PaddleOCR/tools/program.py", line 221, in train preds = model(images) File "/home/jeck/miniconda/envs/paddle_ocr_det/lib/python3.8/site-packages/paddle/fluid/dygraph/layers.py", line 914, in call outputs = self.forward(*inputs, kwargs) File "/home/jeck/Public/PaddleOCR/ppocr/modeling/architectures/base_model.py", line 78, in forward x = self.neck(x) File "/home/jeck/miniconda/envs/paddle_ocr_det/lib/python3.8/site-packages/paddle/fluid/dygraph/layers.py", line 914, in call outputs = self.forward(*inputs, *kwargs) File "/home/jeck/Public/PaddleOCR/ppocr/modeling/necks/pg_fpn.py", line 310, in forward g[4] = self.conv_g4(g[4]) File "/home/jeck/miniconda/envs/paddle_ocr_det/lib/python3.8/site-packages/paddle/fluid/dygraph/layers.py", line 914, in call outputs = self.forward(inputs, kwargs) File "/home/jeck/Public/PaddleOCR/ppocr/modeling/necks/pg_fpn.py", line 63, in forward y = self._conv(inputs) File "/home/jeck/miniconda/envs/paddle_ocr_det/lib/python3.8/site-packages/paddle/fluid/dygraph/layers.py", line 914, in call outputs = self.forward(*inputs, *kwargs) File "/home/jeck/miniconda/envs/paddle_ocr_det/lib/python3.8/site-packages/paddle/nn/layer/conv.py", line 665, in forward out = F.conv._conv_nd( File "/home/jeck/miniconda/envs/paddle_ocr_det/lib/python3.8/site-packages/paddle/nn/functional/conv.py", line 123, in _conv_nd pre_bias = getattr(_C_ops, op_type)(x, weight, attrs) SystemError: (Fatal) Operator conv2d raises an paddle::memory::allocation::BadAlloc exception. The exception content is :ResourceExhaustedError:
Out of memory error on GPU 0. Cannot allocate 112.563965MB memory on GPU 0, 11.665466GB memory has been allocated and available memory is only 99.937500MB.
Please check whether there is any other process using GPU 0.
- If yes, please stop them, or start PaddlePaddle on another GPU.
-
If no, please decrease the batch size of your model.
(at /paddle/paddle/fluid/memory/allocation/cuda_allocator.cc:79) . (at /paddle/paddle/fluid/imperative/tracer.cc:221) `