2025-05-07 20:43:18 | ERROR | data_prepare:get_and_save_daily_data:139 - {'action': 'validate_data', 'file': 'long_hld', 'date': '2025-05-06 00:00:00+08:00', 'status': 'incomplete', 'missing_ratio': '0.41'}
2025-05-07 20:43:18 | CRITICAL | data_prepare:gen_daily_data:349 - {'action': 'gen_daily_data', 'status': 'failed', 'error': 'long_hld data is not complete for 2025-05-06 00:00:00+08:00'}
2025-05-07 20:43:18 | ERROR | data_prepare:gen_daily_data:350 - 生成每日数据过程中发生严重错误
Traceback (most recent call last):
File "/root/project/auto/auto/data_prepare.py", line 357, in
gen_daily_data()
└
File "/root/project/auto/auto/log_utils.py", line 92, in wrapper
return func(*args, **kwargs)
│ │ └ {}
│ └ ()
└
> File "/root/project/auto/auto/data_prepare.py", line 269, in gen_daily_data
rank_long_hld = get_and_save_daily_data('long_hld', trading_day_path, xd_cli, database='technical', table='rank20')
│ │ └
│ └ '/root/project/Research1//daily_data//2025-05-07/'
└
File "/root/project/auto/auto/log_utils.py", line 92, in wrapper
return func(*args, **kwargs)
│ │ └ {'database': 'technical', 'table': 'rank20'}
│ └ ('long_hld', '/root/project/Research1//daily_data//2025-05-07/', )
└
File "/root/project/auto/auto/data_prepare.py", line 146, in get_and_save_daily_data
raise ValueError(f'{file_name} data is not complete for {data.index[-1]}')
ValueError: long_hld data is not complete for 2025-05-06 00:00:00+08:00
future_1d/main_daily/switch
select date, comd, switch from main_daily where method='turnover'
switch last_date is 2025-05-07 00:00:00+08:00
future_1d/main_daily/history_factor
select date, comd, history_factor from main_daily where method='turnover'
history_factor last_date is 2025-05-07 00:00:00+08:00
future_1d/main_daily/volume
select date, comd, volume from main_daily where method='turnover'
volume last_date is 2025-05-07 00:00:00+08:00
future_1d/main_daily/settle
select date, comd, settle from main_daily where method='turnover'
settle last_date is 2025-05-07 00:00:00+08:00
future_1d/main_daily/pre_settle
select date, comd, pre_settle from main_daily where method='turnover'
pre_settle last_date is 2025-05-07 00:00:00+08:00
future_1d/main_daily/contract
select date, comd, contract from main_daily where method='turnover'
contract last_date is 2025-05-07 00:00:00+08:00
future_1d/main_daily/factor_multiply
select date, comd, factor_multiply from main_daily where method='turnover'
factor_multiply last_date is 2025-05-07 00:00:00+08:00
future_1d/main_daily/close
select date, comd, close from main_daily where method='turnover'
close last_date is 2025-05-07 00:00:00+08:00
future_1d/main_daily/opencol
select date, comd, opencol from main_daily where method='turnover'
opencol last_date is 2025-05-07 00:00:00+08:00
future_1d/main_daily/turnover
select date, comd, turnover from main_daily where method='turnover'
turnover last_date is 2025-05-07 00:00:00+08:00
future_1d/main_daily/high
select date, comd, high from main_daily where method='turnover'
high last_date is 2025-05-07 00:00:00+08:00
future_1d/main_daily/low
select date, comd, low from main_daily where method='turnover'
low last_date is 2025-05-07 00:00:00+08:00
future_1d/main_daily/open_interest
select date, comd, open_interest from main_daily where method='turnover'
open_interest last_date is 2025-05-07 00:00:00+08:00
future_1d/future_spot/near_contract_price
select date, comd, near_contract_price from future_spot
near_contract_price last_date is 2025-05-07 00:00:00+08:00
future_1d/future_spot/dominant_contract_price
select date, comd, dominant_contract_price from future_spot
dominant_contract_price last_date is 2025-05-07 00:00:00+08:00
future_1d/future_spot/near_basis
select date, comd, near_basis from future_spot
near_basis last_date is 2025-05-07 00:00:00+08:00
future_1d/future_spot/dom_basis
select date, comd, dom_basis from future_spot
dom_basis last_date is 2025-05-07 00:00:00+08:00
future_1d/future_spot/near_basis_rate
select date, comd, near_basis_rate from future_spot
near_basis_rate last_date is 2025-05-07 00:00:00+08:00
future_1d/future_spot/dom_basis_rate
select date, comd, dom_basis_rate from future_spot
dom_basis_rate last_date is 2025-05-07 00:00:00+08:00
technical/rank20/long_hld
select date, comd, long_hld from rank20
long_hld last_date is 2025-05-06 00:00:00+08:00
2025-05-07 23:44:34 | CRITICAL | data_prepare:gen_daily_data:349 - {'action': 'gen_daily_data', 'status': 'failed', 'error': "Cannot save file into a non-existent directory: '/root/project/Research1/daily_data'"}
2025-05-07 23:44:34 | ERROR | data_prepare:gen_daily_data:350 - 生成每日数据过程中发生严重错误
Traceback (most recent call last):
File "/root/project/auto/auto/data_prepare.py", line 357, in
gen_daily_data()
└
File "/root/project/auto/auto/log_utils.py", line 92, in wrapper
return func(*args, **kwargs)
│ │ └ {}
│ └ ()
└
> File "/root/project/auto/auto/data_prepare.py", line 234, in gen_daily_data
meta_mapping.to_parquet(meta_mapping_path)
│ │ └ '/root/project/Research1//daily_data//meta_mapping.parquet'
│ └
└ id comd_name exchange sector
comd
AP 1 苹果 CZCE Agriculture
CF...
File "/usr/local/lib/python3.10/site-packages/pandas/util/_decorators.py", line 333, in wrapper
return func(*args, **kwargs)
│ │ └ {}
│ └ ( id comd_name exchange sector
│ comd
│ AP 1 苹果 CZCE Agriculture
│ C...
└
File "/usr/local/lib/python3.10/site-packages/pandas/core/frame.py", line 3113, in to_parquet
return to_parquet(
└
File "/usr/local/lib/python3.10/site-packages/pandas/io/parquet.py", line 480, in to_parquet
impl.write(
│ └
└
File "/usr/local/lib/python3.10/site-packages/pandas/io/parquet.py", line 198, in write
path_or_handle, handles, filesystem = _get_path_or_handle(
└
File "/usr/local/lib/python3.10/site-packages/pandas/io/parquet.py", line 140, in _get_path_or_handle
handles = get_handle(
└
File "/usr/local/lib/python3.10/site-packages/pandas/io/common.py", line 749, in get_handle
check_parent_directory(str(handle))
│ └ '/root/project/Research1//daily_data//meta_mapping.parquet'
└
File "/usr/local/lib/python3.10/site-packages/pandas/io/common.py", line 616, in check_parent_directory
raise OSError(rf"Cannot save file into a non-existent directory: '{parent}'")
OSError: Cannot save file into a non-existent directory: '/root/project/Research1/daily_data'
2025-05-07 23:44:34 | CRITICAL | data_prepare::362 - {'action': 'data_prepare', 'status': 'failed', 'error': "[Errno 2] No such file or directory: '/root/project/Research1//daily_data//meta_mapping.parquet'"}
2025-05-07 23:44:34 | ERROR | data_prepare::363 - 数据预处理过程中发生严重错误
Traceback (most recent call last):
> File "/root/project/auto/auto/data_prepare.py", line 359, in
COMD_DICT, SECTOR_DICT, EXCHANGE_DICT = generate_and_save_dicts()
└
File "/root/project/auto/auto/log_utils.py", line 92, in wrapper
return func(*args, **kwargs)
│ │ └ {}
│ └ ()
└
File "/root/project/auto/auto/data_prepare.py", line 33, in generate_and_save_dicts
meta_mapping = pd.read_parquet(f'{DAILY_DATA_PATH}/meta_mapping.parquet')
│ └
└
File "/usr/local/lib/python3.10/site-packages/pandas/io/parquet.py", line 667, in read_parquet
return impl.read(
│ └
└
File "/usr/local/lib/python3.10/site-packages/pandas/io/parquet.py", line 267, in read
path_or_handle, handles, filesystem = _get_path_or_handle(
└
File "/usr/local/lib/python3.10/site-packages/pandas/io/parquet.py", line 140, in _get_path_or_handle
handles = get_handle(
└
File "/usr/local/lib/python3.10/site-packages/pandas/io/common.py", line 882, in get_handle
handle = open(handle, ioargs.mode)
│ │ └ 'rb'
│ └ IOArgs(filepath_or_buffer='/root/project/Research1//daily_data//meta_mapping.parquet', encoding='utf-8', mode='rb', compressi...
└ '/root/project/Research1//daily_data//meta_mapping.parquet'
FileNotFoundError: [Errno 2] No such file or directory: '/root/project/Research1//daily_data//meta_mapping.parquet'