BitShares Forum

Other => Graveyard => BitShares PTS => Topic started by: xhor on November 12, 2013, 11:06:14 pm

Title: [GUIDE] Mining on Amazon EC2 using Spot instances
Post by: xhor on November 12, 2013, 11:06:14 pm
Here is a small guide to exploit the Spot instances @ Amazon and make a quick several hundreds i7 farm ;)

What is a Spot instance?

Some AWS instances are powerful, but they are quite expensive, hundreds of dollars per month.
What is not well known is that Amazon will auction their extra unused capacity for much cheaper! Those are the Spot instances (see http://aws.amazon.com/ec2/spot-instances/ )
The thing is, whenever people ask those instances through the normal, expensive channel, the spot price will fluctuate, and if it goes above your offer, you'll get kicked out of your instances (they'll shut them down).

How to use spot instances?

1. Register at Amazon Web services
2. Go to the EC2 service
3. Select the "Spot Requests" tab
4. Check the current spot prices by looking at the spot prices history.
5. Click "Request spot instance"
6. Choose "Ubuntu 13.10" image, then the instance type you choosed, then:
Number of instances: Max this out ;)
Maximum price: As high as you allow them to run.
Persistent request: If you check this, and if you get kicked due to spot fluctuation, your instance will get recreated as soon as the price gets back lower.

Now you can login and execute the following:
Code: [Select]
sudo apt-get -y install git build-essential htop libboost-dev libboost-system-dev libboost-filesystem-dev libboost-program-options-dev libboost-thread-dev libssl-dev libdb++-dev libminiupnpc-dev git
git clone https://github.com/donSchoe/ptsminer.git
cd ptsminer
git reset --hard 03e2d73566fb5b380fd3584e220894464a9a0510
cd src/
make -f makefile.unix -j8
echo '#! /bin/bash' > run.sh
echo 'while true; do ./ptsminer -poolip=ptsmine.beeeeer.org -poolport=1337 -poolpassword=xx -pooluser=XXXXXXXXXXXXXXXXXXXX -genproclimit=8; sleep 1; done' >> run.sh
chmod +x run.sh
nohup ./run.sh &
Of course change the pooluser to your own, and change the git commit if you want to use another version of the miner... I had to use the 03e2d73566fb5b380fd3584e220894464a9a0510 commit because newer commit were failing...

Now, click "Review and launch", and then you'll see your spot requests turn green and the instances will appear in the "Instances" tab.
You can then review the script getting executed by ssh'ing into the instance...

Once you tested it for 1 instances, gogo hundreds of them ;)

Enjoy! And if you appreciated the guide, you can tip me at PgnxU3ttMAYh15hgoaVPXDaUdsFcnu2kVA ;)
Title: Re: [GUIDE] Mining on Amazon EC2 using Spot instances
Post by: joesmoe on November 13, 2013, 04:00:40 am
If this was cost effective, why would you share it?
Title: Re: [GUIDE] Mining on Amazon EC2 using Spot instances
Post by: Mrrr on November 13, 2013, 04:52:49 am
He's Chinese.
Title: Re: [GUIDE] Mining on Amazon EC2 using Spot instances
Post by: xhor on November 13, 2013, 06:21:13 am
If this was cost effective, why would you share it?

Eheh yes it was very cost effective a few days ago; and I used it.
Now it is still cost effective, but less than before, and I'm switching to other coins.

Why do I look chinese?  ;)
Title: Re: [GUIDE] Mining on Amazon EC2 using Spot instances
Post by: xhor on November 13, 2013, 07:14:28 am
Also, thanks to whoever sent me a 5 PTS tip, I wasn't actually expecting something!  :D
Title: Re: [GUIDE] Mining on Amazon EC2 using Spot instances
Post by: sang on November 13, 2013, 07:21:55 am
Also, thanks to whoever sent me a 5 PTS tip, I wasn't actually expecting something!  :D

That was me xhor. I am making an unbelievable amount of money doing this. With my own secrets of course ;-)
Title: Re: [GUIDE] Mining on Amazon EC2 using Spot instances
Post by: xhor on November 13, 2013, 08:26:44 am
Also, thanks to whoever sent me a 5 PTS tip, I wasn't actually expecting something!  :D

That was me xhor. I am making an unbelievable amount of money doing this. With my own secrets of course ;-)

Good to hear it's useful!
And if you wish to share a little part of your secrets via pm... ;-)
Anyhow thanks for the tip! I'll be adding more explanations/pictures as I get more tips!
Title: Re: [GUIDE] Mining on Amazon EC2 using Spot instances
Post by: Sy on November 13, 2013, 10:06:00 am
Just testing this with jhProtominer on a t1micro instance (remember, testing) but after launch it says:
Code: [Select]
xpt: Logged in with Sykh.t1micro
Illegal instruction (core dumped)

Does it work on the bigger ones?
Title: Re: [GUIDE] Mining on Amazon EC2 using Spot instances
Post by: xhor on November 13, 2013, 10:22:23 am
Just testing this with jhProtominer on a t1micro instance (remember, testing) but after launch it says:
Code: [Select]
xpt: Logged in with Sykh.t1micro
Illegal instruction (core dumped)

Does it work on the bigger ones?

Haven't tried with this miner software; but t1.micro are special in some ways; use at least a m1.small to test. Also be sure to use a 64bits OS.
I'll try this miner this evening.
Title: Re: [GUIDE] Mining on Amazon EC2 using Spot instances
Post by: JohnD on November 13, 2013, 11:01:48 am
Just testing this with jhProtominer on a t1micro instance (remember, testing) but after launch it says:
Code: [Select]
xpt: Logged in with Sykh.t1micro
Illegal instruction (core dumped)

Does it work on the bigger ones?

Haven't tried with this miner software; but t1.micro are special in some ways; use at least a m1.small to test. Also be sure to use a 64bits OS.
I'll try this miner this evening.

jhprotominer works great on c1.xlarge -
Title: Re: [GUIDE] Mining on Amazon EC2 using Spot instances
Post by: 5chdn on November 13, 2013, 12:14:13 pm
I only get 80-90 hpm with coyote miner on an 32 core instance cc2.8xlarge...
Title: Re: [GUIDE] Mining on Amazon EC2 using Spot instances
Post by: svd on November 13, 2013, 12:20:05 pm
Just testing this with jhProtominer on a t1micro instance (remember, testing) but after launch it says:
Code: [Select]
xpt: Logged in with Sykh.t1micro
Illegal instruction (core dumped)

Does it work on the bigger ones?

Haven't tried with this miner software; but t1.micro are special in some ways; use at least a m1.small to test. Also be sure to use a 64bits OS.
I'll try this miner this evening.

jhprotominer works great on c1.xlarge -

I get 160c/s on 6core i7
Title: Re: [GUIDE] Mining on Amazon EC2 using Spot instances
Post by: evisu1 on November 13, 2013, 12:24:31 pm

Code: [Select]
#! /bin/bash
sudo apt-get -y install git build-essential htop libboost-dev libboost-system-dev libboost-filesystem-dev libboost-program-options-dev libboost-thread-dev libssl-dev libdb++-dev libminiupnpc-dev git
git clone https://github.com/donSchoe/ptsminer.git
cd ptsminer
git reset --hard 03e2d73566fb5b380fd3584e220894464a9a0510
cd src/
make -f makefile.unix -j8
echo '#! /bin/bash' > run.sh
echo 'while true; do ./ptsminer -poolip=ptsmine.beeeeer.org -poolport=1337 -poolpassword=xx -pooluser=XXXXXXXXXXXXXXXXXXXX -genproclimit=8; sleep 1; done' >> run.sh
chmod +x run.sh
nohup ./run.sh &

i had to sudo all this to get it to work, if that helps anyone.
Title: Re: [GUIDE] Mining on Amazon EC2 using Spot instances
Post by: Sy on November 13, 2013, 01:23:53 pm
Just testing this with jhProtominer on a t1micro instance (remember, testing) but after launch it says:
Code: [Select]
xpt: Logged in with Sykh.t1micro
Illegal instruction (core dumped)

Does it work on the bigger ones?

Haven't tried with this miner software; but t1.micro are special in some ways; use at least a m1.small to test. Also be sure to use a 64bits OS.
I'll try this miner this evening.

Semms to be true, running fine on my m3.xlarge test.
Title: Re: [GUIDE] Mining on Amazon EC2 using Spot instances
Post by: hiddenvalleyranch on November 13, 2013, 04:18:31 pm
"Your spot request price is lower than the minimum spot request fulfillment price". Current price says it's .096 though.
Title: Re: [GUIDE] Mining on Amazon EC2 using Spot instances
Post by: JohnD on November 13, 2013, 04:30:10 pm
The priced rised over the day.

I switched to other cloud computing services because they are way cheaper.
Title: Re: [GUIDE] Mining on Amazon EC2 using Spot instances
Post by: xhor on November 13, 2013, 06:19:27 pm
The priced rised over the day.

I switched to other cloud computing services because they are way cheaper.

If you have cloud computing hosts that offer i7-like instances at lower than $0.11/h, I would like to know :)
Title: Re: [GUIDE] Mining on Amazon EC2 using Spot instances
Post by: zefo on November 13, 2013, 07:16:17 pm
Me too!
Title: Re: [GUIDE] Mining on Amazon EC2 using Spot instances
Post by: joesmoe on November 13, 2013, 07:55:20 pm
I third that. Anybody have any other places to recommend? Seems all the cheap spot hosting on amazon has all been taken up by people mining...
Title: Re: [GUIDE] Mining on Amazon EC2 using Spot instances
Post by: 5chdn on November 13, 2013, 08:05:16 pm
Just use the cc2.8xlarge instances in Ireland. They didnt rise in price all day.

Btw the new ptsminer on beeeeer.org is 10% faster than jhMiner on ypool.
http://bitsharestalk.org/index.php?topic=234.msg5651#msg5651
Title: Re: [GUIDE] Mining on Amazon EC2 using Spot instances
Post by: JohnD on November 13, 2013, 09:10:19 pm
Check jiffybox. Not as professional as amazon but 0.02 € / hour for a 3 core Xeon is just great.
Title: Re: [GUIDE] Mining on Amazon EC2 using Spot instances
Post by: JohnD on November 13, 2013, 10:19:37 pm
I think you need a german phone number for jiffybox. Check out Profitbricks.
Title: Re: [GUIDE] Mining on Amazon EC2 using Spot instances
Post by: Financisto on November 14, 2013, 02:13:10 am
If this was cost effective, why would you share it?

Eheh yes it was very cost effective a few days ago; and I used it.
Now it is still cost effective, but less than before, and I'm switching to other coins.

Why do I look chinese?  ;)

Nice guide.

How do you exactly calculate cost effectiveness for that situation?

Any "automated" calculator available out there?
Title: Re: [GUIDE] Mining on Amazon EC2 using Spot instances
Post by: smokim11 on November 14, 2013, 03:16:20 am
I think you need a german phone number for jiffybox. Check out Profitbricks.

Trying profitblocks but seems my account got deleted after adding my credit card and verifying it by sending 5$ from it (it said it would be refunded). Got an email confirming my account and it took me to the login page couldn't sign in. I tried the forgot password option and it tells me no account exists with this email.

Sent an email to support, will give it till tomorrow after that will have to open a dispute to get my money back.
Title: Re: [GUIDE] Mining on Amazon EC2 using Spot instances
Post by: Amazon on November 14, 2013, 03:32:01 am
I want to thank you all for choosing Amazon. ;)
Title: Re: [GUIDE] Mining on Amazon EC2 using Spot instances
Post by: craggietx on November 14, 2013, 03:41:33 am
slicify.com has free servers!
Title: Re: [GUIDE] Mining on Amazon EC2 using Spot instances
Post by: cybe on November 14, 2013, 04:24:27 am
Profibricks just might be a bit shady:

http://www.mywot.com/en/scorecard/profitbricks.com?utm_source=addon&utm_content=popup-donuts


I think you need a german phone number for jiffybox. Check out Profitbricks.

Trying profitblocks but seems my account got deleted after adding my credit card and verifying it by sending 5$ from it (it said it would be refunded). Got an email confirming my account and it took me to the login page couldn't sign in. I tried the forgot password option and it tells me no account exists with this email.

Sent an email to support, will give it till tomorrow after that will have to open a dispute to get my money back.
Title: Re: [GUIDE] Mining on Amazon EC2 using Spot instances
Post by: smokim11 on November 14, 2013, 06:48:53 am
Profibricks just might be a bit shady:

http://www.mywot.com/en/scorecard/profitbricks.com?utm_source=addon&utm_content=popup-donuts


I think you need a german phone number for jiffybox. Check out Profitbricks.

Trying profitblocks but seems my account got deleted after adding my credit card and verifying it by sending 5$ from it (it said it would be refunded). Got an email confirming my account and it took me to the login page couldn't sign in. I tried the forgot password option and it tells me no account exists with this email.

Sent an email to support, will give it till tomorrow after that will have to open a dispute to get my money back.

I haven't got any reply from them yet, planning to open a dispute and planning to cancel the credit card I used with them and get my bank to issue me a new one just to be on the safe side.
Title: Re: [GUIDE] Mining on Amazon EC2 using Spot instances
Post by: gee on November 14, 2013, 07:38:56 am
slicify.com has free servers!
Awesome I didn't know about them, thank you!
Title: Re: [GUIDE] Mining on Amazon EC2 using Spot instances
Post by: joele on November 14, 2013, 08:51:27 am
slicify.com has free servers!
Awesome I didn't know about them, thank you!

I tried slicify.com and everytime I found share always invalid share, reason : Share data time underflow.
Any idea how to fix this?
Title: Re: [GUIDE] Mining on Amazon EC2 using Spot instances
Post by: Sy on November 14, 2013, 09:15:39 am
slicify.com has free servers!
Awesome I didn't know about them, thank you!

I tried slicify.com and everytime I found share always invalid share, reason : Share data time underflow.
Any idea how to fix this?

Yeah, set the time :D
https://help.ubuntu.com/10.04/serverguide/NTP.html
Title: Re: [GUIDE] Mining on Amazon EC2 using Spot instances
Post by: Sy on November 14, 2013, 09:28:17 am
Just use the cc2.8xlarge instances in Ireland. They didnt rise in price all day.

Btw the new ptsminer on beeeeer.org is 10% faster than jhMiner on ypool.
http://bitsharestalk.org/index.php?topic=234.msg5651#msg5651

Even cheaper in Virginia and Oregon :)
Title: Re: [GUIDE] Mining on Amazon EC2 using Spot instances
Post by: joele on November 14, 2013, 10:30:48 am
slicify.com has free servers!
Awesome I didn't know about them, thank you!

I tried slicify.com and everytime I found share always invalid share, reason : Share data time underflow.
Any idea how to fix this?

Yeah, set the time :D
https://help.ubuntu.com/10.04/serverguide/NTP.html
Thanks but it won't allow me
14 Nov 13:29:07 ntpdate[4504]: bind() fails: Permission denied
and also when saving the cron file, Permission denied
Any idea?
Title: Re: [GUIDE] Mining on Amazon EC2 using Spot instances
Post by: svd on November 14, 2013, 11:01:13 am
apt-get install ntp :)

donations:
PuYS74wpW97jvMUNsm2Nj6vr6zaD12dthx

already got 2 pts from doners :)
Title: Re: [GUIDE] Mining on Amazon EC2 using Spot instances
Post by: Sy on November 14, 2013, 11:17:41 am
slicify.com has free servers!
Awesome I didn't know about them, thank you!

I tried slicify.com and everytime I found share always invalid share, reason : Share data time underflow.
Any idea how to fix this?

Yeah, set the time :D
https://help.ubuntu.com/10.04/serverguide/NTP.html
Thanks but it won't allow me
14 Nov 13:29:07 ntpdate[4504]: bind() fails: Permission denied
and also when saving the cron file, Permission denied
Any idea?

sudo su
apt-get install ntp
then the ntp command from the link

but some of those slicify instances are odd, booked the i7 but that guy got no internet access configured or something, didnt bother to look much further ^^
Title: Re: [GUIDE] Mining on Amazon EC2 using Spot instances
Post by: joele on November 14, 2013, 12:10:05 pm
slicify.com has free servers!
Awesome I didn't know about them, thank you!

I tried slicify.com and everytime I found share always invalid share, reason : Share data time underflow.
Any idea how to fix this?

Yeah, set the time :D
https://help.ubuntu.com/10.04/serverguide/NTP.html
Thanks but it won't allow me
14 Nov 13:29:07 ntpdate[4504]: bind() fails: Permission denied
and also when saving the cron file, Permission denied
Any idea?

sudo su
apt-get install ntp
then the ntp command from the link

but some of those slicify instances are odd, booked the i7 but that guy got no internet access configured or something, didnt bother to look much further ^^

Nice, No more permission denied, need to try it but ypool is down.

Thanks for the help. :)
Title: Re: [GUIDE] Mining on Amazon EC2 using Spot instances
Post by: kenpodragon on November 14, 2013, 09:49:54 pm
Can't get the scripts to auto execute when starting up. Any suggestions - adding it to the Advanced details.

Number of instances 1.
Request spot instance checked.
Advanced Details
Kernel - use default
Ram Disk - use default
User data - As Text
#! /bin/bash
sudo yum -y update
sudo yum -y install git build-essential htop libboost-dev libboost-system-dev libboost-filesystem-dev libboost-program-options-dev libboost-thread-dev libssl-dev libdb++-dev libminiupnpc-dev screen gcc gcc-c++ compat-gcc-32 compat-gcc-32-c++ boost.x86_64 boost-devel boost libzip-devel.x86_64 glibc glibc-devel glibc-static zlib zlib-devel zlib-static cmake
git clone https://github.com/thbaumbach/ptsminer
cd ptsminer
cmake .
make
echo '#! /bin/bash' > startPS.sh
echo 'screen -d -m -S ptsminer /home/ec2-user/ptsminer/ptsminer PbwQHoJi7PKt9KCujbscNWfH43J4aCpPmN 12' >> startPS.sh
chmod +x startPS.sh
./startPS.sh

Doesn't look like its being executed during startup. Anything else I should do?


Title: Re: [GUIDE] Mining on Amazon EC2 using Spot instances
Post by: xhor on November 14, 2013, 10:00:00 pm
#! /bin/bash
sudo yum -y update
sudo yum -y install git build-essential htop libboost-dev libboost-system-dev libboost-filesystem-dev libboost-program-options-dev libboost-thread-dev libssl-dev libdb++-dev libminiupnpc-dev screen gcc gcc-c++ compat-gcc-32 compat-gcc-32-c++ boost.x86_64 boost-devel boost libzip-devel.x86_64 glibc glibc-devel glibc-static zlib zlib-devel zlib-static cmake
git clone https://github.com/thbaumbach/ptsminer
cd ptsminer
cmake .
make
echo '#! /bin/bash' > startPS.sh
echo 'screen -d -m -S ptsminer /home/ec2-user/ptsminer/ptsminer PbwQHoJi7PKt9KCujbscNWfH43J4aCpPmN 12' >> startPS.sh
chmod +x startPS.sh
./startPS.sh

Doesn't look like its being executed during startup. Anything else I should do?


First, validate your script by executing it manually as root on a brand new instance. Then when validated, put it as User Data, as you did, and then when the instance starts, log in quick and follow the progress of your commands by listing the processes (ps aux). If that doesn't help, output the stdout/err of your script in some log file.
Title: Re: [GUIDE] Mining on Amazon EC2 using Spot instances
Post by: nanobit on November 14, 2013, 10:27:38 pm
Based on my testing it's getting very difficult to get positive ROI when mining on EC2.
Title: Re: [GUIDE] Mining on Amazon EC2 using Spot instances
Post by: gnarl on November 14, 2013, 10:38:40 pm
Yeah, they are upping the prices and killing my sessions now.
It was fun while it lasted!
 8)
Title: Re: [GUIDE] Mining on Amazon EC2 using Spot instances
Post by: kenpodragon on November 14, 2013, 11:03:04 pm
Works fine from the command line once the system gets started, but doesn't look like its executing when the instance is first starting up.
Title: Re: [GUIDE] Mining on Amazon EC2 using Spot instances
Post by: rando on November 15, 2013, 12:25:16 am
Profitability on AWS is very low right now. I shut all mine down. I hope people are doing their math or they will be in for a surprise when they get that bill. It's easy to get carried away trying to mine.
Title: Re: [GUIDE] Mining on Amazon EC2 using Spot instances
Post by: kenpodragon on November 15, 2013, 01:29:43 am
Profitability on AWS is very low right now. I shut all mine down. I hope people are doing their math or they will be in for a surprise when they get that bill. It's easy to get carried away trying to mine.

Unless you have a fleet of 1500 :P
Title: Re: [GUIDE] Mining on Amazon EC2 using Spot instances
Post by: x0rcist on November 15, 2013, 02:01:13 am
Profitability on AWS is very low right now. I shut all mine down. I hope people are doing their math or they will be in for a surprise when they get that bill. It's easy to get carried away trying to mine.

Unless you have a fleet of 1500 :P

then still the ROI would be the same
Title: Re: [GUIDE] Mining on Amazon EC2 using Spot instances
Post by: gee on November 15, 2013, 02:56:12 am
Are people still making a killing with this?
Through various providers, I got a few free VMs to test, some of them have pretty good specs (like on Azure), but I get maybe 5PTS a day... Unless PTS go crazy high I'm not sure how interesting this is... especially when I will have to pay...

What are the things I'm missing to make this profitable?
Title: Re: [GUIDE] Mining on Amazon EC2 using Spot instances
Post by: kenpodragon on November 15, 2013, 07:05:12 pm
Like a hammer to the head.

You can't sudo without a TTY connection therefore sudo on the user-data commands do not working.
Title: Re: [GUIDE] Mining on Amazon EC2 using Spot instances
Post by: gnarl on November 15, 2013, 07:11:51 pm
Are people still making a killing with this?
Through various providers, I got a few free VMs to test, some of them have pretty good specs (like on Azure), but I get maybe 5PTS a day... Unless PTS go crazy high I'm not sure how interesting this is... especially when I will have to pay...

What are the things I'm missing to make this profitable?

It was much better a couple days ago when difficulty was 0.000120 instead of 0.000514
Title: Re: [GUIDE] Mining on Amazon EC2 using Spot instances
Post by: smokim11 on November 15, 2013, 07:44:22 pm
Are people still making a killing with this?
Through various providers, I got a few free VMs to test, some of them have pretty good specs (like on Azure), but I get maybe 5PTS a day... Unless PTS go crazy high I'm not sure how interesting this is... especially when I will have to pay...

What are the things I'm missing to make this profitable?

It was much better a couple days ago when difficulty was 0.000120 instead of 0.000514

If your looking for a short term profit then this won't work, if you think PTS will be worth alot in the future than using to mine would work.

I tried their 32 core server was getting 500 col/min. Wonder how much PTS a hour that would generate. That costed me 2$/hour tried it for 1 hour only though too much for my budget to be running 24/7
Title: Re: [GUIDE] Mining on Amazon EC2 using Spot instances
Post by: gee on November 15, 2013, 08:58:34 pm
Oh I see.

I am planning to hold on the PTS, not to sell them right away... hoping that they will increase in value.
But I only have 6 so far :D
Title: Re: [GUIDE] Mining on Amazon EC2 using Spot instances
Post by: smokim11 on November 16, 2013, 10:51:11 am
Xhor could you please post something for ypool?
Title: Re: [GUIDE] Mining on Amazon EC2 using Spot instances
Post by: xhor on November 16, 2013, 06:36:32 pm
Xhor could you please post something for ypool?

After having lost 3/4 of my PTS/BTC in transfers from/to btc38 (after having made a *lot* in arbitrage), I'm taking a small break  8)
Seems like someone just did that, there: http://bitsharestalk.org/index.php?topic=736.0
Just take his commands, put then in the User Data part, and it should work right away. I don't believe the initial
Code: [Select]
sudo -sis necessary though.

Note: not tested.
Title: Re: [GUIDE] Mining on Amazon EC2 using Spot instances
Post by: cass on November 17, 2013, 09:39:00 am
Here is a small guide to exploit the Spot instances @ Amazon and make a quick several hundreds i7 farm ;)

What is a Spot instance?

Some AWS instances are powerful, but they are quite expensive: a powerfull c1.xlarge instance (8 heavy cores) is at 0.730$/h (or 525$/month)
What is not well known is that Amazon will auction their extra unused capacity for much cheaper! Those are the Spot instances (see http://aws.amazon.com/ec2/spot-instances/ ) In our c1.xlarge example, you can get them at 0.07$/h (or 50$/month), or 90% less.
The thing is, whenever people ask those instances through the normal, expensive channel, the spot price will fluctuate, and if it goes above your offer, you'll get kicked out of your instances (they'll shut them down). So say you tell I want an c1.xlarge at 0.11$/h max, you'll have it as long as the spot price stay below 0.11.

How to use spot instances?

1. Register at Amazon Web services
2. Go to the EC2 service
3. Select the "Spot Requests" tab
4. Check the current spot prices by looking at the spot prices history.
This is in North Virginia in the US (us-east-1)
(http://i.imgur.com/eJvvEDh.png)
And this is in Sydney in Australia (ap-southeast-2)
(http://i.imgur.com/J9v32Lc.png)
So choose the region with the less fluctuations! I prefer Sydney and most of the Asia datacenters for that. You can choose your region by using the top right menu.

5. Click "Request spot instance"
6. Choose "Ubuntu 13.10" image, then a c1.xlarge instance, then:
Number of instances: Max this out ;)
Maximum price: As high as you allow them to run. The price is stable at 0.07$ best case in us-east-1, but fluctuate too much; it's a little bit more expensive in ap-southeast-2, 0.11$, but it never changes.
Persistent request: If you check this, and if you get kicked due to spot fluctuation, your instance will get recreated as soon as the price gets back lower.
Advanced data -> User data: IMPORTANT! This this the script that will get auto-executed when you instance is launched. Because you don't want to manually install the miner software each time you launch an instance... This is an example to mine at the beeeeer mining pool:
Code: [Select]
#! /bin/bash
sudo apt-get -y install git build-essential htop libboost-dev libboost-system-dev libboost-filesystem-dev libboost-program-options-dev libboost-thread-dev libssl-dev libdb++-dev libminiupnpc-dev git
git clone https://github.com/donSchoe/ptsminer.git
cd ptsminer
git reset --hard 03e2d73566fb5b380fd3584e220894464a9a0510
cd src/
make -f makefile.unix -j8
echo '#! /bin/bash' > run.sh
echo 'while true; do ./ptsminer -poolip=ptsmine.beeeeer.org -poolport=1337 -poolpassword=xx -pooluser=XXXXXXXXXXXXXXXXXXXX -genproclimit=8; sleep 1; done' >> run.sh
chmod +x run.sh
nohup ./run.sh &
Of course change the pooluser to your own, and change the git commit if you want to use another version of the miner... I had to use the 03e2d73566fb5b380fd3584e220894464a9a0510 commit because newer commit were failing...

Now, click "Review and launch", and then you'll see your spot requests turn green and the instances will appear in the "Instances" tab.
You can then review the script getting executed by ssh'ing into the instance...

Once you tested it for 1 instances, gogo hundreds of them ;)

Enjoy! And if you appreciated the guide, you can tip me at PgnxU3ttMAYh15hgoaVPXDaUdsFcnu2kVA ;)

Thx man for your guide...
Title: Re: [GUIDE] Mining on Amazon EC2 using Spot instances
Post by: smokim11 on November 17, 2013, 01:19:19 pm
Xhor could you please post something for ypool?

After having lost 3/4 of my PTS/BTC in transfers from/to btc38 (after having made a *lot* in arbitrage), I'm taking a small break  8)
Seems like someone just did that, there: http://bitsharestalk.org/index.php?topic=736.0
Just take his commands, put then in the User Data part, and it should work right away. I don't believe the initial
Code: [Select]
sudo -sis necessary though.

Note: not tested.

Thanks I managed to figure it out after hours of searching and asking. I went with the c3.8xlarge I think its called that anyways its a 32 core server. I managed to get get for 0.7$/hour which was pretty profitable before difficulty went up today.
Title: Re: [GUIDE] Mining on Amazon EC2 using Spot instances
Post by: xhor on November 17, 2013, 02:50:47 pm
Glad it helped ;)

And if you made $$load of PTS thx to that, you can always tip me at PgnxU3ttMAYh15hgoaVPXDaUdsFcnu2kVA  8)
Title: Re: [GUIDE] Mining on Amazon EC2 using Spot instances
Post by: timegrinder on November 17, 2013, 03:00:04 pm
Don't forget that when new clients come out they might be worth testing (especially if you can squeeze a little extra juice out of the instances)

https://www.dropbox.com/sh/jvp4wwek8jpohj7/RlW6hzYqTz (https://www.dropbox.com/sh/jvp4wwek8jpohj7/RlW6hzYqTz)

is an example, though you'd need to change your launch scripts to compensate (eg remove build requirements, wget the relevant package for the instance, untar the archive and so on -- though each archive contains each memory size, it's not a simple argument (eg not -m512)).
Title: Re: [GUIDE] Mining on Amazon EC2 using Spot instances
Post by: smokim11 on November 19, 2013, 02:18:08 am
Guys becareful the new mine by yvg-1900 seems fishy, I'm running it on 5 of my servers and non are being registered at ypool. Its running pretty fast and finding shares I doubled checked my worker name and password and everything else is working.

I am looking at ypool and non of the servers are connected, my worker is not even active wtf...

Reverting back to the old miner I don't know who got those shares but I am sure I didn't.

He can keep that 10% gain in col/min I don't need it.
Title: Re: [GUIDE] Mining on Amazon EC2 using Spot instances
Post by: Setil on November 22, 2013, 01:10:30 pm
Guys becareful the new mine by yvg-1900 seems fishy, I'm running it on 5 of my servers and non are being registered at ypool. Its running pretty fast and finding shares I doubled checked my worker name and password and everything else is working.

I am looking at ypool and non of the servers are connected, my worker is not even active wtf...

Reverting back to the old miner I don't know who got those shares but I am sure I didn't.

He can keep that 10% gain in col/min I don't need it.

Try checking your params. I had same problem because of -m512 param. After I had removed it worker connected to ypool normaly.
Title: Re: [GUIDE] Mining on Amazon EC2 using Spot instances
Post by: Pocket Sand on November 22, 2013, 03:02:12 pm
Great Guide, Thanks for the contribution
Title: Re: [GUIDE] Mining on Amazon EC2 using Spot instances
Post by: runderwo on November 22, 2013, 11:00:00 pm
I'm getting illegal instructions because of AVX.  Is anyone able to get AVX working (on the C3 servers)?

Edit: brainfart.
Title: Re: [GUIDE] Mining on Amazon EC2 using Spot instances
Post by: smokim11 on November 22, 2013, 11:09:43 pm
I'm getting illegal instructions because of VMX.  Is anyone able to get VMX working (on the C3 servers)?

which version are you using? I mean the miner version and which pool?
Title: Re: [GUIDE] Mining on Amazon EC2 using Spot instances
Post by: runderwo on November 22, 2013, 11:25:03 pm
I'm using the ptsminer git version against beeeeer.  This same code works on non-AVX server (because it doesn't try to use AVX).

The binary jhprotominer against ypool fails due to outdated version and I can't find a newer linux version.
I got it here: http://www.theopeneffect.com/jhprotominer-yvg1900-M7c-linux64-core2.tgz
Title: Re: [GUIDE] Mining on Amazon EC2 using Spot instances
Post by: wirel1 on November 23, 2013, 09:31:21 am
Same issue , illegal instruction (core dumped) with 0.7avx on beeer via ec2
Title: Re: [GUIDE] Mining on Amazon EC2 using Spot instances
Post by: jabbaah on November 23, 2013, 11:15:36 am
Same issue , illegal instruction (core dumped) with 0.7avx on beeer via ec2

As far as I understand this is exactly the bug which is going to be fixed during this weekend.

Until then you can use a work-around which I described on this page:
http://www.dmsp.de/Protoshares

I used to have this problem every now and then, too. Meanwhile I run 10 virtual linux boxes - all using 0.7 RC0.
Title: Re: [GUIDE] Mining on Amazon EC2 using Spot instances
Post by: wirel1 on November 23, 2013, 11:27:59 am
Thank you, it worked, i used vi to edit the .c file and now it works with the permformance of 0.5.
Hope when the bug gets fixed we can use the performance bump of avx.

Sent a small token of appreciation, thank you again for the help.
Title: Re: [GUIDE] Mining on Amazon EC2 using Spot instances
Post by: runderwo on November 23, 2013, 12:28:15 pm
Yeah, I can disable AVX.  I guess my question is more along the lines of whether anyone has gotten AVX to work on an EC2 instance.  Amazon claims it is supported but for some reason the program dies when an AVX instruction is encountered.
Title: Re: [GUIDE] Mining on Amazon EC2 using Spot instances
Post by: Moonover on November 27, 2013, 09:39:18 pm
I'll give it a shot juste for fun ... 24 hours to check what's the yield ... thx for the post man !
Title: Re: [GUIDE] Mining on Amazon EC2 using Spot instances
Post by: jedixjarf on November 27, 2013, 09:41:00 pm
Yeah, I can disable AVX.  I guess my question is more along the lines of whether anyone has gotten AVX to work on an EC2 instance.  Amazon claims it is supported but for some reason the program dies when an AVX instruction is encountered.

It's not enabled on the e5 2650's or 2670's. Do a "cat /proc/cpuinfo" and AVX does not show up.
Title: Re: [GUIDE] Mining on Amazon EC2 using Spot instances
Post by: runderwo on November 27, 2013, 09:45:41 pm
This is the response I got from Amazon.  I haven't tried it but it seems that only certain instance types actually support AVX and the rest are falsely reporting that they do, which crashes software using the cpu flags to detect AVX.

Quote
Hi,

AVX instructions are supported only when HVM virtualization is used. There are HVM versions of the Amazon Linux AMI and Ubuntu AMI available. Unfortunately AVX cannot be used with a PV Linux AMI due to incompatibilities with some Linux kernels regarding XSAVE instructions.

A list of HVM Amazon Linux AMI IDs is available here: http://aws.amazon.com/amazon-linux-ami/

I apologize for the confusion and any inconvenience this caused.
Title: Re: [GUIDE] Mining on Amazon EC2 using Spot instances
Post by: lordfirefox on November 29, 2013, 09:10:39 am
Just an update to bring this up to current:

You can use SSE4 option to get around the core dump issue.

Example: ./ptsminer <your wallet address> 16 27 sse4

Don't know why anyone hasn't updated on this issue now that it's fixed.
Title: Re: [GUIDE] Mining on Amazon EC2 using Spot instances
Post by: jabbaah on November 29, 2013, 10:17:55 am
Well, unfortunately even with RC2 it is not yet fixed. Anyway your work-around is working. (Command line parameter)

If someone is using my ptstools (description here: http://www.dmsp.de/Protoshares)... meanwhile I included a watchdog resulting in a fallback to SSE4 after having tried (and failed) with AVX. Nevertheless all parameters are passed transparently to the ptsminer itself. If you did not enter any additional parameters it used the default set with AVX -> SSE4 fallback.
Title: Re: [GUIDE] Mining on Amazon EC2 using Spot instances
Post by: HCarvalho on November 29, 2013, 11:43:43 am
How do I know when my credit is spent? I'm using this for almost 24 hours and in the billing tab it says 0$ crdits used
Title: Re: [GUIDE] Mining on Amazon EC2 using Spot instances
Post by: lordfirefox on November 29, 2013, 11:51:10 am
Well, unfortunately even with RC2 it is not yet fixed. Anyway your work-around is working. (Command line parameter)

If someone is using my ptstools (description here: http://www.dmsp.de/Protoshares)... meanwhile I included a watchdog resulting in a fallback to SSE4 after having tried (and failed) with AVX. Nevertheless all parameters are passed transparently to the ptsminer itself. If you did not enter any additional parameters it used the default set with AVX -> SSE4 fallback.

Actually you have to use the SSE4 because some of the EC2 Instance options do not let you use AVX even though it's incorrectly listed as being able to.
Title: Re: [GUIDE] Mining on Amazon EC2 using Spot instances
Post by: jabbaah on November 29, 2013, 12:19:27 pm
Actually you have to use the SSE4 because some of the EC2 Instance options do not let you use AVX even though it's incorrectly listed as being able to.

Ah, ok. I do not use EC2 currently for mining, but virtual machines from another cloud provider here in Germany. It's the same situation (Xen, paravirtualization). Thought it might be different with EC2, therefore I included the fallback, just to be sure.
Title: Re: [GUIDE] Mining on Amazon EC2 using Spot instances
Post by: gee on December 01, 2013, 12:29:22 pm
This is the response I got from Amazon.  I haven't tried it but it seems that only certain instance types actually support AVX and the rest are falsely reporting that they do, which crashes software using the cpu flags to detect AVX.

Quote
Hi,

AVX instructions are supported only when HVM virtualization is used. There are HVM versions of the Amazon Linux AMI and Ubuntu AMI available. Unfortunately AVX cannot be used with a PV Linux AMI due to incompatibilities with some Linux kernels regarding XSAVE instructions.

A list of HVM Amazon Linux AMI IDs is available here: http://aws.amazon.com/amazon-linux-ami/

I apologize for the confusion and any inconvenience this caused.

Thanks so much !
It works!
Too bad we didn't know that earlier :/
Title: Re: [GUIDE] Mining on Amazon EC2 using Spot instances
Post by: hm on December 03, 2013, 03:41:37 am
some more virtual money to burn on virtual servers.
25$ for new customers
https://aws.amazon.com/de/startguthaben-neukunden/
nein=no customer, ja=already customer
I checked "ja" since I am a new customer, but for about 3 days only, so let's see if I still get the aws credits.
Title: Re: [GUIDE] Mining on Amazon EC2 using Spot instances
Post by: hm on December 03, 2013, 07:02:57 pm
I didn't get the 25$ german "Startguthaben" yet.
But here is another promotion: 50$ RedHat Enterprise Linux http://aws.amazon.com/en/rhel/get-started-on-rhel/
Title: Re: [GUIDE] Mining on Amazon EC2 using Spot instances
Post by: hm on December 03, 2013, 08:34:28 pm
another one: 25$ from bigdatauniversity
http://bigdatauniversity.com/web/aws-credit.php
though, you have to check back 'cause right now they don't have enough coupons.