Hi @xinxin.li.seattle, Sorry about that!The ondemand_to_spot script creates the .aws.creds file using the same approach as setup_p2.sh (using aws configure get aws_access_key_id and aws configure get aws_secret_access_key). If these were not set when you ran on_demand_to_spot.sh (e.g. you haven't run aws configure), you can create .aws.creds in ec2-spotter using this template:AWSAccessKeyId=XXXXXXXXXXXXXXXXXXXXAWSSecretKey=XXXXXXXXXXXXXXXXXXXXXXXXXX

PS: Also I missed some crucial steps in the existing instance approach, which I just updated.

Thank you this will be quite helpful. I am not a lot familiar with AWS. So this question might seem stupid. It is more sought of clarification. Is the EBS volume running even when the spot instance stops? And if the volume is running, it will cost to keep the EBS volume running right? And how much on average it cost?

Hey @Saiyan! Yes, you pay for the EBS volume regardless of whether it is attached to an instance. Currently it's $0.1/GB-month. This means that if you have a 100GB volume for a full month, it will cost you $10, which IMO is not that much.

@slavivanov Thank you for fixing the script. It is working for me very well!!

One advice is to put a word of caution in approach 1 fresh instance, because in step 3 it terminates not just the instance from Step 1, but all of your existing instances created with fastai setup script. Luckily, I always backup my data and code in the cloud, so nothing is lost. Because p2.xlarge was approved with a limit, for those with a small limit, you want to be very cautious about accidentally terminating your only approved instance. Other than that, this script works exceptionally well and is very easy to follow. I highly recommend it. Great job and thank you for sharing it @slavivanov!

Hey, I've tried setting it all up but I get the following error:ondemand_to_spot.sh: 7: export: i-0278bf10da31b66a9: bad variable nameI suspect some small change in the bash script would do, but I'm still not sure what that should be. Could you please look into that?

Thanks a lot!

EDIT 1:So using a temporary fix (substitution instance id in the script) worked but then that was the output:

I'm not sure what that could be and I'm not sure which variable name from the first issue could be wrong...

EDIT 2:So I managed to fix my first issue (getting instance ID), but I'm still stuck at "ondemand_to_spot.sh: 91: ondemand_to_spot.sh: cannot create ec2-spotter/.aws.creds: Directory nonexistent", even though I created the directory manually...

Hi @slazien, sorry about this!@z0k is exactly right. The ondemand_to_spot file was previously in a different folder. Follow his instructions to get this solved. (I've also pushed a fix for this to github).

Thank you so much for your responses, changing that line (why didn't I notice that myself?) fixed it all. There is still an error when running start_spot.sh (start_spot.sh: 5: start_spot.sh: Bad substitution), but it seems to work fine.

EDIT: so after terminating the on-demand instance and converting it to spot with the script it turns out nvidia-smi is not working, which is strange:

modprobe: ERROR: ../libkmod/libkmod.c:514 lookup_builtin_file() could not open builtin file '/lib/modules/4.4.0-64-generic/modules.builtin.bin'modprobe: ERROR: ../libkmod/libkmod.c:586 kmod_search_moddep() could not open moddep file '/lib/modules/4.4.0-64-generic/modules.dep.bin'modprobe: ERROR: ../libkmod/libkmod.c:586 kmod_search_moddep() could not open moddep file '/lib/modules/4.4.0-64-generic/modules.dep.bin'modprobe: ERROR: ../libkmod/libkmod-module.c:832 kmod_module_insert_module() could not find module by name='nvidia_367'modprobe: ERROR: could not insert 'nvidia_367': Unknown symbol in module, or unknown parameter (see dmesg)NVIDIA-SMI has failed because it couldn't communicate with the NVIDIA driver. Make sure that the latest NVIDIA driver is installed and running.

Also, while trying to apt-get update it says dpkg was interrupted, ugh...

E: dpkg was interrupted, you must manually run 'sudo dpkg --configure -a' to correct the problem.

This is awesome work, well done - It will save me millions over the next few year.

I've spent several hours installing everything and configured it now so the instances launch and worked out how to mount the instance.

One questions is that I don't have jupyter notebook installed, so when I do installed - it routes to localhost.Also the nvidia-smi doesn't seem to work, so I'm wondering if I need to install a bunch of scripts?