Managing memory usage
Dask is a great tool for managing memory usage when processing datasets. Here are few tips on how to utilize dask in python to maximize the compute resource. Check your…
Climatology
How climatological statistics are defined and computed: https://climate.usu.edu/people/yoshi/pyclm101/monthly.html#anomalies-and-climatology
scp permission denied
When transferring files from a server (local) to server (remote), you may encounter permission problem. Recently I had the permission issue with scp . I had the write permission yet…
R resources for earth data
R for Earth-System Science - tutorials on how to read/write netcdf4 files and manipulate geospatial data using netcf4 and terra packages R for Geospatial analysis - shows how to use…
Git: How to get deleted files from previous commit
When you deleted a file and need it back, you can access the file if it was once committed to your git. First, whatever you are doing, make sure to…
python: logging
While one can use if-else statements, the use of logging and raising errors (raise) can be a better way to handle errors and warnings in complex projects for the following…
NetCDF Operators (NCO)
Useful nco commands For full descriptions, check out the resource links. nccopy The nccopy compresses a netCDF file at various levelsusage: nccopy [-flag] [original nc filename] [compressed nc filename]example: nccopy…
Update chartjs htmlwidget
Sources: https://www.htmlwidgets.org/develop_intro.html https://www.chartjs.org/docs/latest/samples/area/line-datasets.html inst/htmlwidgets/ # directory to add css, js, yaml, libraries # To modify chartjs high level, go to chartjs/R/chartjs.R # To modify each line of graphs chartjs/R/series.R #…
How to install miniconda on MacOS
source: https://docs.anaconda.com/free/miniconda/ # create parent directory miniconda3 mkdir -p ~/miniconda3 # download curl https://repo.anaconda.com/miniconda/Miniconda3-latest-MacOSX-arm64.sh -o ~/miniconda3/miniconda.sh # execute installation and updates bash ~/miniconda3/miniconda.sh -b -u -p ~/miniconda3 # remove setup…
Conda cheatsheet
# CREATE CONDA ENVIRONMENT conda create -n [MY_ENV_NAME] # create conda environment conda create --name [MY_ENV_NAME] python=3.5 # create conda env with py version conda conda create -f env.yaml #…
Github: resolving conflict
Conflict 1: hint: Updates were rejected because a pushed branch tip is behind its remote hint: counterpart. Check out this branch and integrate the remote changes hint: (e.g. 'git pull…
R – here package
https://here.r-lib.org/articles/here.html it is a useful package to set project root directory # my project structure is following: #> /home/sunny/work/myproject #> ├── data_proc #> │ └── mycode.R #> ├── analysis #>…
How to create a website from Quarto
source: https://quarto.org/docs/publishing/github-pages.html Download and install Quarto CLI from https://quarto.org/docs/download/ Open your terminal and go to your working directory (e.g. ~/projects/sunny-bak/) Create a quarto scaffolding for your project (template) called funsite;…
How to set up multiple ssh keys (accounts) in multiple workspaces
https://blog.gitguardian.com/8-easy-steps-to-set-up-multiple-git-accounts/ Create ssh keys using key-gen (id_personal, id_prof) Add keys using ssh-agent Create config file in ~/.ssh/config # ~/ssh/config Host * AddKeysToAgent yes UseKeychain yes IdentityFile ~/.ssh/<personal_key> Host * AddKeysToAgent…
Useful R codes
How to get min and max value and the associated year that min and max values occurred previous <- dfile %>%filter(YearLabel == "Previous") %>%group_by(MonthName) %>%summarize(min_extent = min(extent_mil),max_extent = max(extent_mil),year_min =…