Push notifications in your browser are not yet configured.
You are not logged in, you may not see all content and functionalities. If you have an account, please login .
ncrcat command regarding sftp server
Dear colleagues, I have an issue with remote processing of my COSMO - CLM output data. I have a remote NAS (Network Attached Storage), there are many COSMO - CLM output data (many lffd*.nc files) and there is no possibility to install NCO at NAS . This NAS has an sftp address and connection, I can connect it using simple sftp in Linux (sftp login_name@ip_address, then password). However, what I would like to do, is to postprocess these lffd*.nc data remotely, e.g., I would like to run NCO command (e.g., ncrcat) on my local computer to get the ncrcat-ted file locally using the sftp access to my NAS . I have read the NCO sourceforge Userguide and tried many variants, however, I haven’t any success.
I have tried the following commands (and many similar ones) and got the following results:
ncrcat -h -O -v T_2M,lat,lon -l . sftp://192.168.1.58:/DATA/OUT_Arctic_5.05_final_1996/out02/lffd1996*.nc -o T_2M.nc
vplatonv@192.168.1.58’s password:
Connected to 192.168.1.58.
Multiple source paths, but destination “./lffd1996*.nc” is not a directory
ncrcat: ERROR Synchronous fetch command failed
ncrcat -h -O -v T_2M,lat,lon -l . vplatonv@192.168.1.58:/DATA/OUT_Arctic_5.05_final_1996/out02/lffd*.nc -o T_2M.nc
vplatonv@192.168.1.58’s password:
scp: /DATA/OUT_Arctic_5.05_final_1996/out02/lffd*.nc: No such file or directory
ncrcat: ERROR Synchronous fetch command failed
ncrcat -h -O -v T_2M,lat,lon -H -p sftp://192.168.1.58:/DATA/OUT_Arctic_5.05_final_1996/out02 -l . lffd*.nc -o T_2M.nc
vplatonv@192.168.1.58’s password:
Connected to 192.168.1.58.
Multiple source paths, but destination “./lffd*.nc” is not a directory
ncrcat: ERROR Synchronous fetch command failed
ncrcat -h -O -v T_2M,lat,lon -p sftp://192.168.1.58:/DATA/OUT_Arctic_5.05_final_1996/out02 -l . lffd*.nc -o T_2M.nc
vplatonv@192.168.1.58’s password:
Connected to 192.168.1.58.
Multiple source paths, but destination “./lffd*.nc” is not a directory
ncrcat: ERROR Synchronous fetch command failed
ncrcat -h -O -v T_2M,lat,lon sftp://192.168.1.58:/DATA/OUT_Arctic_5.05_final_1996/out02/* -l . -o T_2M.nc
usage: sftp [-46aCfpqrv] [-B buffer_size] [-b batchfile] [-c cipher] [-D sftp_server_path] [-F ssh_config] [-i identity_file] [-J destination] [-l limit] [-o ssh_option] [-P port] [-R num_requests] [-S program] [-s subsystem | sftp_server] destination
ncrcat: ERROR Synchronous fetch command failed
ncrcat -h -O -v T_2M,lat,lon -l . sftp://192.168.1.58:/DATA/OUT_Arctic_5.05_final_1996/out02/* -o T_2M.nc
usage: sftp [-46aCfpqrv] [-B buffer_size] [-b batchfile] [-c cipher] [-D sftp_server_path] [-F ssh_config] [-i identity_file] [-J destination] [-l limit] [-o ssh_option] [-P port] [-R num_requests] [-S program] [-s subsystem | sftp_server] destination
ncrcat: ERROR Synchronous fetch command failed
The following commands (and many similar ones) didn’t get any results and any errors:
ncrcat -h -O -v T_2M,lat,lon -l . sftp://192.168.1.58:/DATA/OUT_Arctic_5.05_final_1996/out02/lffd*.nc
ncrcat -h -O -v T_2M,lat,lon -H -p sftp://192.168.1.58:/DATA/OUT_Arctic_5.05_final_1996/out02/lffd*.nc -l . -o T_2M.nc
ncrcat -h -O -v T_2M,lat,lon -l . sftp://192.168.1.58:/DATA/OUT_Arctic_5.05_final_1996/out02/lffd1996010100.nc
There is the only command which worked somehow:
ncrcat -h -O -v T_2M,lat,lon -l . sftp://192.168.1.58:/DATA/OUT_Arctic_5.05_final_1996/out02/lffd1996010100.nc -o T_2M.nc
vplatonv@192.168.1.58’s password:
Connected to 192.168.1.58.
Fetching /DATA/OUT_Arctic_5.05_final_1996/out02/lffd1996010100.nc to ./lffd1996010100.nc
/DATA/OUT_Arctic_5.05_final_1996/out02/lffd1996010100.nc 52% 61MB 3.9MB/s 00:14 ETA
However, there is not necessary to operate the single lffd file, I need to operate multiple files. Excuse me for a lot of examples, code and logs.
So, does anyone have some experience in operating NCO via sftp having NCO on local machine and aiming to operate remote machine without NCO ? I would be very grateful for any hints in this case. Thanks a lot!
I have never tried this. Since it is very NCO specific, you may pose your question on the web site
https://sourceforge.net/p/nco/discussion/9830/
I have also no idea how that could work, but is there a reason why you don’t mount your NAS on your computer and then access the files directly?
Dear colleagues, thanks for your replies and recommendations, I have posted this question on the NCO Sourceforge Forum. My reason to organize NAS was to share COSMO - CLM data with other users, which is much more useful via sftp access. Moreover, I have a personal notebook, which is not applicable for stationary direct access the files on NAS . Therefore, I have organized NAS and SFTP access to data on NAS , and would like to operate these data remotely using NCO .
Dear Vladimir,
Usually I put file list as argument to the commands, for example in bash I generate an array of file names:
#loop after all files – make array of directory+files names
for ((i=0;i
and the array is argument to the command:
ncecat ${member_files[*]} /users/kdm/plgajacz/tmp/cat_tmp
but I see in NCO documentation different solution:
/bin/ls | grep ${CASEID}_’……’.nc | xargs -x ncecat -o foo.nc
Cheers
Adam
Dear colleagues, thank you for feedback and help. I have partially solved my problem and would like to share this solution for further users and applicatons. Detailed answer and useful links were provided by Charlie Zender on the NCO forum: [[https://sourceforge.net/p/nco/discussion/9830/thread/74d094c06c/?limit=25]]. One of links contained a hint by Adam, thanks a lot!