2025-07-08 19:05:02 | INFO | data_prepare::345 - {'action': 'data_prepare', 'status': 'started'}
2025-07-08 19:05:02 | INFO | data_prepare:gen_daily_data:230 - {'action': 'gen_daily_data', 'status': 'started'}
2025-07-08 19:05:06 | INFO | data_prepare:gen_daily_data:236 - {'action': 'connect_client', 'status': 'success'}
2025-07-08 19:05:06 | INFO | data_prepare:gen_daily_data:238 - {'action': 'fetch_meta_mapping', 'status': 'started'}
2025-07-08 19:05:06 | INFO | data_prepare:gen_daily_data:251 - {'action': 'get_trading_day_path', 'status': 'attempt'}
2025-07-08 19:05:06 | INFO | data_prepare:get_trading_day_path:64 - {'action': 'get_trading_day_path', 'status': 'started'}
2025-07-08 19:05:06 | INFO | data_prepare:get_trading_day_path:110 - {'action': 'get_trading_day_path', 'status': 'completed', 'path': '/root/data/Research1//daily_data//2025-07-08/'}
2025-07-08 19:05:06 | INFO | data_prepare:gen_daily_data:254 - {'action': 'fetch_data', 'status': 'started', 'category': 'basic_trading'}
2025-07-08 19:05:06 | INFO | data_prepare:get_and_save_daily_data:120 - {'action': 'get_save_data', 'file': 'switch', 'status': 'started'}
2025-07-08 19:05:08 | INFO | data_prepare:get_and_save_daily_data:131 - {'action': 'process_data', 'file': 'switch', 'last_date': '2025-07-08 00:00:00+08:00'}
2025-07-08 19:05:08 | INFO | data_prepare:get_and_save_daily_data:155 - {'action': 'get_save_data', 'file': 'switch', 'status': 'completed', 'date': '2025-07-08 00:00:00+08:00'}
2025-07-08 19:05:08 | INFO | data_prepare:get_and_save_daily_data:120 - {'action': 'get_save_data', 'file': 'history_factor', 'status': 'started'}
2025-07-08 19:05:11 | INFO | data_prepare:get_and_save_daily_data:131 - {'action': 'process_data', 'file': 'history_factor', 'last_date': '2025-07-08 00:00:00+08:00'}
2025-07-08 19:05:11 | INFO | data_prepare:get_and_save_daily_data:155 - {'action': 'get_save_data', 'file': 'history_factor', 'status': 'completed', 'date': '2025-07-08 00:00:00+08:00'}
2025-07-08 19:05:11 | INFO | data_prepare:get_and_save_daily_data:120 - {'action': 'get_save_data', 'file': 'volume', 'status': 'started'}
2025-07-08 19:05:13 | INFO | data_prepare:get_and_save_daily_data:131 - {'action': 'process_data', 'file': 'volume', 'last_date': '2025-07-08 00:00:00+08:00'}
2025-07-08 19:05:13 | ERROR | data_prepare:get_and_save_daily_data:142 - {'file': 'volume', 'date': '2025-07-08 00:00:00+08:00', 'status': 'incomplete', 'missing_ratio': '0.34'}
2025-07-08 19:05:13 | CRITICAL | data_prepare:gen_daily_data:339 - {'action': 'gen_daily_data', 'status': 'failed', 'error': 'volume data is not complete for 2025-07-08 00:00:00+08:00'}
2025-07-08 19:05:13 | ERROR | data_prepare:gen_daily_data:340 - 生成每日数据过程中发生严重错误
Traceback (most recent call last):
File "/root/project/auto/auto/data_prepare.py", line 347, in
gen_daily_data()
└
> File "/root/project/auto/auto/data_prepare.py", line 257, in gen_daily_data
volume = get_and_save_daily_data('volume', trading_day_path, xd_cli, latest_trading_day=latest_trading_day)
│ │ │ └ datetime.date(2025, 7, 8)
│ │ └
│ └ '/root/data/Research1//daily_data//2025-07-08/'
└
File "/root/project/auto/auto/data_prepare.py", line 148, in get_and_save_daily_data
raise ValueError(f'{file_name} data is not complete for {data.index[-1]}')
ValueError: volume data is not complete for 2025-07-08 00:00:00+08:00
2025-07-08 19:05:13 | INFO | data_prepare:generate_and_save_dicts:27 - {'action': 'generate_dicts', 'status': 'started'}
2025-07-08 19:05:13 | CRITICAL | data_prepare::352 - {'action': 'data_prepare', 'status': 'failed', 'error': "'未分类'"}
2025-07-08 19:05:13 | ERROR | data_prepare::353 - 数据预处理过程中发生严重错误
Traceback (most recent call last):
> File "/root/project/auto/auto/data_prepare.py", line 349, in
COMD_DICT, SECTOR_DICT, EXCHANGE_DICT = generate_and_save_dicts()
└
File "/root/project/auto/auto/data_prepare.py", line 47, in generate_and_save_dicts
sector_dict = meta_mapping['sector'].apply(lambda x: sector_range[x]).to_dict()
│ └ {'Agriculture': 0, 'Ferrous': 1, 'Finance': 2, 'Energy': 3, 'Precious': 4, 'NonFerrous': 5}
└ id comd_name exchange sector
comd
AP 1 苹果 CZCE Agriculture...
File "/usr/local/lib/python3.10/site-packages/pandas/core/series.py", line 4935, in apply
).apply()
File "/usr/local/lib/python3.10/site-packages/pandas/core/apply.py", line 1422, in apply
return self.apply_standard()
│ └
└
File "/usr/local/lib/python3.10/site-packages/pandas/core/apply.py", line 1502, in apply_standard
mapped = obj._map_values(
│ └
└ comd
AP Agriculture
CF Agriculture
CJ Agriculture
CY Agriculture
FG Ferrous
...
PR E...
File "/usr/local/lib/python3.10/site-packages/pandas/core/base.py", line 925, in _map_values
return algorithms.map_array(arr, mapper, na_action=na_action, convert=convert)
│ │ │ │ │ └ True
│ │ │ │ └ None
│ │ │ └ . at 0x7f67b0085900>
│ │ └ array(['Agriculture', 'Agriculture', 'Agriculture', 'Agriculture',
│ │ 'Ferrous', 'Finance', 'Finance', 'Finance', 'Financ...
│ └
└
File "/usr/local/lib/python3.10/site-packages/pandas/core/algorithms.py", line 1743, in map_array
return lib.map_infer(values, mapper, convert=convert)
│ │ │ │ └ True
│ │ │ └ . at 0x7f67b0085900>
│ │ └ array(['Agriculture', 'Agriculture', 'Agriculture', 'Agriculture',
│ │ 'Ferrous', 'Finance', 'Finance', 'Finance', 'Financ...
│ └
└
File "pandas/_libs/lib.pyx", line 2999, in pandas._libs.lib.map_infer
File "/root/project/auto/auto/data_prepare.py", line 47, in
sector_dict = meta_mapping['sector'].apply(lambda x: sector_range[x]).to_dict()
│ │ └ '未分类'
│ └ {'Agriculture': 0, 'Ferrous': 1, 'Finance': 2, 'Energy': 3, 'Precious': 4, 'NonFerrous': 5}
└ '未分类'
KeyError: '未分类'
2025-07-08 19:05:13 | INFO | data_prepare::355 - {'action': 'data_prepare', 'status': 'completed'}
future_1d/main_daily/switch
select date, comd, switch from main_daily where method='turnover'
future_1d/main_daily/history_factor
select date, comd, history_factor from main_daily where method='turnover'
future_1d/main_daily/volume
select date, comd, volume from main_daily where method='turnover'