How To Use Kogence

Jump to: navigation, search

Detailed Instructions for Using Octave on Kogence

In order to use Octave on Kogence free supercomputing platform, you can either fork and modify an existing Octave public project or alternatively you can also start from scratch and create a new Octave project. See detailed instructions below.

Fork and Modify an Existing Octave Project

Step 1:

Kogence hosts 30+ public Octave projects. See the list at the bottom of this page or click here. Click on any of the projects you like to open the associated "project dashboard". Here you can see the detailed description of the model. This is also a WYSIWYG editable page. Feel free to edit the description as you like by clicking at several "edit" links next to each section title on the page.

Step 2:

On the top you will see a toolbar that provides all the functionality to fork, modify and execute the project on supercomputing cloud with your choices of hardware and software resources. You will also be able to control privacy and collaboration settings. If you click on the "Files" tab seen on the top-left corner of the top toolbar, you will be taken to "project code listing" page like the one shown below in the snapshot.

Project Code Listing2.png

Step 3:

If you click on the "copy" button in the toolbar, as shown in the snapshot below, you would be able to fork the project to make a personal copy for yourself.

Fork Project.png

Step 4:

If you click on any text formatted file in the list of files (such as elk.in, *.m etc), that file will open on a syntax highlighted code editor, as shown below in the snapshot, in your browser window itself.

Code Editor.png

You can modify the code as you want and then save the changes in your personal copy of the project. This approach can be used to modify projects that are based on software that have a text based input file (such as Elk, Octave, Matlab, MEEP, MPB, REMS, S4, MESH etc.) to define and execute problems.

Kogence also natively supports software that are based on a Graphical User Interface (GUI) such as jupyterNotebook, Comsol, Lumerical, ParaView, VESTA, Ansys, Abaqus etc. If your project is based on a GUI based software then you will edit and modify your project after starting the "Kogence Interactive Remote Display" in Step 9 below.

If you click on the folder icon on the top far-left corner, as shown in the snapshot below, you will browse back to the page that shows the list of all files in the project.

File Listing.png

Step 5:

If you click on the "collaborate" button on the toolbar, as shown in the snapshot below, you will be able to choose privacy settings, add collaborators and assign rights/permissions to them. You can choose to make some collaborators "admin" for this project. Those collaborators will now be able to add/delete collaborators and assign them permissions. They would also be able to further fork the project and potentially change the privacy settings to make it a public project. To prevent these actions you should make the collaborators "user" and not "admin".

Collaboration Settings2.png

Step 6:

If you click on the "settings" button on the toolbar, as shown in the snapshot below, you will be able to choose the type of cloud hardware/machine on which you want to execute your project. You will see two sub-tabs -- one for machine selection and other for software selection. First click on the machine sub-tab.

Machine Settings2.png

Step 7:

The second sub-tab allows you to select the set of software you want to execute on the cloud.

Software Settings2.png

In the snapshot shown, we have elected to have an xterm available so we can do some miscellaneous post-processing. We put the & in the textbox to run the xterm in the background. Simultaneously, we have chosen to execute our Elk project. Once Elk project execution is done, we have chosen to start Octave software so we can postprocess and chart the simulation results.

For a JupyterNotebook project, for example, you would select "JupyterNotebook" from the dropdown menu and you will put the name of the notebook (*.ipynb) in the textbox.

Step 8:

Then press the last button (the "play" button) on the top right toolbar to execute the project on free cloud supercomputing platform.

Run.png

Step 9:

Look at the messages in the log panel and the bottom of your browser window.

Log Window3.png

Once it says that visualizer is ready, you can then click on the "display" button on the top toolbar, as shown in the snapshot above, to connect to the "Kogence Interactive Remote Display" on the browser itself (see the snapshot below). This is Kogence proprietary real time interactive remote display and it is expected to give you same experience as if you were on your local desktop. You can then post-process /chart your data as you would like.

RemoteDisplay.PNG

Step 10:

Any graphical outputs that you save in formats such as jpg, png, bmp etc. would be automatically pushed to the "project dashboard" page. Dashboard pages are WYSIWYG editable. You can modify the text, define the project/problem and rearrange the results as you like. Project dashboards have same permissions as those defined by you for the whole project. For example, a private project dashboard will only show up in the search queries of your collaborators only.


Create a New Octave Project from Scratch Step 1:

Log into Kogence. You will see a list of your own projects (empty list to begin with). You can click on the Model Library link on the top toolbar and you will see some Example Models to get you started. You can copy/fork one of the Example Models and follow the steps as described in the previous Section.

Alterntaively, you can also create a new model starting from scratch. Click on the "+" button on top left corner, as shown in the snapshot below, to create a new project.

FirstPage.png

Add a title, a brief description and choose if you want to make the project public or private.

Step 2:

You will now see a screen like one shown below.

Add File2.png

Click on the "+" button on top left corner again to add or upload new file. If you upload a zip file, Kogence will automatically unzip it. You can now follow Step 4 onward as described in previous Section.

Combining Multiple Software in Workflow

A typical scientific simulation workflow involves three steps. STEP 1: pre-processing using CAD environment on master node. STEP2: Sending solver jobs to multiple worker nodes in non-graphical batch mode. STEP3: Post-processing suing CAD environment on master node. Furthermore, pre-processing, post-processing and solvers may all be different software packages. Kogence Stack tab allows you to configure all such complex workflows at ease. First you will select CAD programs, such as pwgui. These will come up in master node. Next you can add solvers like pw.x and send those to cluster for parallel computing. Nect you can add software like Gnuplot, XCrysDen, VESTA , ParaView, Matlab or Octave for post processing and charting your simulation results.

Blocking vs No Blocking Executions: By default all software selected in Stack tab work in blocking mode. Meaning second software/solver will not start until first one finished. If you want them to get started simultaneously then check mark the Run With Previous checkbox next to the command in the Stack tab of your model.

Please note that on Kogence all software and solvers are deployed on docker containers. You can bring your own docker container and deploy it on Kogence. Once you select the containers you want, Kogence automatically composes them so you can call one program from another just like you would on your onprem workstation or on your desktop. Therefore, before you start calling programs in shell terminal or in your bash script make sure you selected all software and solvers that your bash script needs otherwise those solvers would not be available to be used in the bash script. You can skip automatically invoking them through Kogence Stack tab by not selecting the Entrypoint Binary. This way you can make them available to your bash script and then invoke them from your bash script.

Output and Error Logs

On Kogence, stdout is automatically saved under Files tab of your model in a file called ___titusiOut. Similarly, stderr is automatically saved in another file called ___titusiError. While your model is running, you will see Visualizer button being active in the top right corner of the NavBar. You will see errors and outputs being printed live on the screen. Once simulation ends and machine/cluster terminates, the Visualizer will become inactive. Updated ___titusiOut and___titusiError files will automatically sync back to the app webserver and you will can view these updated files under the Files tab of your model.

Model Input and Result Files

You can upload and edit your input files under Files tab of your model. Similarly,, model outputs and results are also saved under Files tab of your model. Please make sure that you upload and edit your model files before launching the simulation. Once simulation is launched, editing and uploading under Files tab is locked and Kogence web app is connected with high spared NFS on both the master and the worker nodes. So your data is automatically shared in live mode between all your master and worker nodes. Once simulation has been launched, and if requested a terminal shell program such as xterm or gnome-terminal under software->settings, then you can use the shell terminal to check all your model inputs and outputs in you job folder. After simulation ends, the lock on the Files tab would be removed and you can see the updated files under the Files tab again.