{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Overview\n", "\n", "There are four possible providers of compute services, or consulting services on campus:\n", "\n", "- [Cornell Center of Social Sciences (CCSS)](https://socialsciences.cornell.edu/computing-and-data/cloud-computing-solutions) offers general purpose and secure computing services.\n", "- [Center for Advanced Computing (CAC)](https://www.cac.cornell.edu/about/Default.aspx) offers advanced and cloud computing solutions, including in the [national (ACCESS-CI)](https://access-ci.org/) and [state (NYSTAR)](http://esd.ny.gov/nystar/) computing infrastructure \n", "- [Biotechnology Resource Center (BioHPC)](https://www.biotech.cornell.edu/facilities-brc) offers advanced on-campus computing solutions (including managing the Econ Department's own cluster)\n", "- [Cornell Information Technology (CIT)](https://it.cornell.edu/) is the general Cornell IT department, but can also engineer solutions for cloud services (Cornell has framework subscriptions to [Amazon Web Service (AWS/EC2)](https://it.cornell.edu/cornell-cloud/amazon-web-services-aws-contract) and [Microsoft cloud services (Azure)](https://it.cornell.edu/cornell-cloud/cornell-azure-contract)).\n" ] }, { "cell_type": "code", "execution_count": 11, "metadata": { "tags": [ "full-width", "remove-input" ] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Project root directory: /mnt/local/raid5_1/vilhuber/Workspace-non-encrypted/git/LDI/ecco-notes\n" ] }, { "data": { "text/html": [ "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
SystemOSmaxNodeRAMmaxNodecorestotalRAMtotalcoresSharedgoodforCostFurther Info Link
CCSSWindows256GB4about 1024GBabout 16YesDesktop computing plus some GPUFreeAvailable software
BioHPC reservedLinux1024GB323177GB360Nolow-power chunked HPCFreeDescription on BioHPC website
BioHPC SLURMLinuxup to 1024GBup to 32up to 1129 GBup to 144Nolow-power fine-grained HPCFreeSee info below
BioHPC generalLinux1024GB56not surenot sureNoHPC some GPUFeeFull list of hardware
CAC Red CloudLinux + Windows240GB128not knownnot knownNoHPC + some GPUFeeMore info
DigitalOceanLinux256GB48unlimitedunlimitedNoHPC + some GPUFeeMore info
AWSLinux + Windows1024GB128unlimitedunlimitedNoHPC + some GPUFeeMore info
AzureLinux + Windows3800GB416unlimitedunlimitedNoHPC + some GPUFeeMore info
" ], "text/plain": [ "" ] }, "execution_count": 11, "metadata": {}, "output_type": "execute_result" } ], "source": [ "import pandas as pd\n", "import os\n", "from IPython.display import HTML\n", "\n", "\n", "def find_project_root(start_dir='.', file_to_find='favicon.ico'):\n", " \"\"\"\n", " Find the project root directory by searching for a specific file.\n", " \n", " Args:\n", " start_dir (str): The directory to start the search from.\n", " file_to_find (str): The name of the file to search for.\n", " \n", " Returns:\n", " str: The absolute path of the project root directory.\n", " \"\"\"\n", " current_dir = os.path.abspath(start_dir)\n", " \n", " while True:\n", " # Check if the file exists in the current directory\n", " if os.path.isfile(os.path.join(current_dir, file_to_find)):\n", " return current_dir\n", " \n", " # Check if we've reached the root directory\n", " parent_dir = os.path.dirname(current_dir)\n", " if parent_dir == current_dir:\n", " raise FileNotFoundError(f\"Could not find '{file_to_find}' in the directory tree.\")\n", " \n", " # Move up one directory\n", " current_dir = parent_dir\n", "\n", "# Example usage\n", "project_root = find_project_root()\n", "#print(f\"Project root directory: {project_root}\")\n", "\n", "\n", "\n", "# Read the CSV file\n", "df = pd.read_csv(os.path.join(project_root,\"_data\",\"eccochoice.csv\"))\n", "\n", "# Create a new column with the hyperlink\n", "df['Further Info Link'] = \"\" + df['Further info'] + \"\"\n", "\n", "# Drop the original columns if you don't need them in the table\n", "df = df.drop(['Further info', 'Further info URL'], axis=1)\n", "\n", "# Convert the DataFrame to an HTML table\n", "table = df.to_html(index=False, \n", " classes='table table-striped table-bordered table-sm', escape=False, \n", " render_links=True)\n", "\n", "\n", "# Render the HTML table in Jupyter Notebook\n", "HTML(table)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "The table is sorted by increasing complexity of preparation and cost (which are collinear). A starting point on how to easily use the latter cloud resources in a (relatively) easy way are\n", "\n", "- the nice [tutorial by Andrew Heiss](https://www.andrewheiss.com/blog/2018/07/30/disposable-supercomputer-future/) \n", "- the [analogsea](https://github.com/pachadotdev/analogsea) and [future](https://cran.r-project.org/web/packages/future/index.html) R packages." ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.11.8" }, "widgets": { "application/vnd.jupyter.widget-state+json": { "state": {}, "version_major": 2, "version_minor": 0 } } }, "nbformat": 4, "nbformat_minor": 4 }