Author Topic: Open source optimized PTS CPU miner (BETA)  (Read 47689 times)

0 Members and 1 Guest are viewing this topic.

Offline rots

  • Newbie
  • *
  • Posts: 12
    • View Profile
Re: Open source optimized PTS CPU miner (BETA)
« Reply #101 on: February 01, 2014, 01:06:22 pm »
Any plan to release windows bineary file?

Following up on yvg1900's release of yam, I figured I'd improve the state of the art of the open source versions a bit:

https://github.com/dave-andersen/ptsminer

I haven't made it build yet on windows (it just needs to compile the avx2 assembly code - should be straightforward if someone wants to clue me in on how to appropriately invoke gcc there), but it should work on other platforms.  As a warning, I've only really tried it on avx2, since I'm a fan of Haswell.  THIS SOFTWARE SHOULD BE CONSIDERED A BETA QUALITY RELEASE.  At best. 

As with my GPU release, this one is based very directly on ptsminer, so it's tied to beeeeer for the moment.  I plan to fix that and let it be used with other pools in the near future, but that's going to take some dev work.  sigh.

There's a lot of optimization to be done, but this gets the basics as far as memory subsystem optimization, and bridges a lot of the gap between the old OSS version and yam M7i.  I haven't tried out M7j, mind you -- it's probably a bit faster still, but this release should bridge the gap considerably.

It incorporates the same optional, extendible 1% dev fee that the gpu miner does.  Prior ptsminer devs, if you feel like you should be in the list, please PM me and I'll get you added!

With gratitude to FreeTrade for the donation that kept me interested in hacking on and releasing this stuff, and to yvg1900 for some very engaging unofficial competition. *grin*

  -Dave

Offline dga

  • Full Member
  • ***
  • Posts: 122
    • View Profile
Re: Open source optimized PTS CPU miner (BETA)
« Reply #100 on: January 30, 2014, 11:04:27 am »
Hi dga,

After 3 or 4 days, the v8 avx slow down from 320 to 260 cpm.
v8 avx2 not sure since run as daemon, but profit seems drop to 75%。

kill the ptsminer and restart seems fix it.

I'm not sure what's happening, since the yam get a lot of reject from beeeeer.org too.

yam on 1GH has no problem using new xpt2h protocol and port 18120.

Now ptsminer is beeeeer.org lock-in.

Any plan to support xpt2h protocol?

I haven't seen avx2 slow down, but beeeeer has had a bad string of luck lately with block finding - my profit is also down a fair bit. 

My own avx beta8 client hasn't slowed down:

2014-Jan-25 17:00:19 | 760.0 c/m | 14.1 sh/m
2014-Jan-30 06:00:07 | 774.9 c/m | 12.2 sh/m

but that doesn't mean there's not something wrong.  What CPU are you running the avx one on and with how many threads?

I do hope to add more protocol support.  I have real work taking up all of my time until this weekend, but I'll check out xpt2h then.

  -Dave

Offline ptsrush

  • Full Member
  • ***
  • Posts: 84
    • View Profile
Re: Open source optimized PTS CPU miner (BETA)
« Reply #99 on: January 30, 2014, 12:19:43 am »
Hi dga,

After 3 or 4 days, the v8 avx slow down from 320 to 260 cpm.
v8 avx2 not sure since run as daemon, but profit seems drop to 75%。

kill the ptsminer and restart seems fix it.

I'm not sure what's happening, since the yam get a lot of reject from beeeeer.org too.

yam on 1GH has no problem using new xpt2h protocol and port 18120.

Now ptsminer is beeeeer.org lock-in.

Any plan to support xpt2h protocol?

Offline dga

  • Full Member
  • ***
  • Posts: 122
    • View Profile
Re: Open source optimized PTS CPU miner (BETA)
« Reply #98 on: January 26, 2014, 12:40:56 am »
haswell e3-1230 v3 upgraded.

beta7 avx2 cpm : 530
beta8 avx2 cpm : 542

It's now faster than my 530 cpm GT 560!

by the way, seems ptsminer work 60 s for developer then 200 s for miner.
next round 1200 s for developer and 40,000 s for miner.

That's good if I'm running a server and never power off.

But when I run ptsminer on my desktop, in most case I run about 20,000 s
then power off.

So 1200 / 20000 = 6%.

for avx2 even 6% the ptsminer still far better than yam, so just for your information.

Glad to hear it's running well on the E3.

Noted about the devmine fee.  I'll fix that in a few betas.  What it really should be is an exponentially increasing sequence (with a cap) -- dev 60, user 2000;  dev 120, user 4000;  dev 240, user 8000; etc., which would reduce the problem you're seeing if you kill at exactly the wrong time, while still reducing the amount of interruption due to mining switches.  There are a few other things I want to do to make the dev mining more robustly fair under disconnects/etc., which is why I haven't just thrown out the exponential version.

Offline ptsrush

  • Full Member
  • ***
  • Posts: 84
    • View Profile
Re: Open source optimized PTS CPU miner (BETA)
« Reply #97 on: January 25, 2014, 11:20:41 pm »
haswell e3-1230 v3 upgraded.

beta7 avx2 cpm : 530
beta8 avx2 cpm : 542

It's now faster than my 530 cpm GT 560!

by the way, seems ptsminer work 60 s for developer then 200 s for miner.
next round 1200 s for developer and 40,000 s for miner.

That's good if I'm running a server and never power off.

But when I run ptsminer on my desktop, in most case I run about 20,000 s
then power off.

So 1200 / 20000 = 6%.

for avx2 even 6% the ptsminer still far better than yam, so just for your information.
« Last Edit: January 26, 2014, 12:32:26 am by ptsrush »

Offline dga

  • Full Member
  • ***
  • Posts: 122
    • View Profile
Re: Open source optimized PTS CPU miner (BETA)
« Reply #96 on: January 25, 2014, 08:18:14 pm »
I've placed beta8 for haswell/avx2 and now for avxsse online.  It fixes (I believe) the bug that was causing failures when not enough hugepages were available, and incorporates the latest round of speed improvements for Haswell/AVX2.  Speed isn't changed much for avxsse, but I'm narrowing in on some more general improvements that should help there too.

http://www.cs.cmu.edu/~dga/ptsminer/

This is a quite worthwhile upgrade for the Haswell/AVX2 crowd.  Expect at least a 20cpm jump and probably more - I'm still letting the cpm benchmarks run, but my dev benchmarks suggest somewhere between a 5-10% speedup over beta7.  I'll update this post tonight with some actual CPM numbers from an i7-4770.

Update 2:
Totally rough guesstimate:

[STATS] 2014-Jan-25 17:34:57 | 570.1 c/m | 8.9 sh/m | VL: 299 (99.7%), RJ: 1 (0.3%), ST: 0 (0.0%)

I expect sustained rates of 565 c/m over a longer period of time.  Not bad, little CPU, not bad.

  -Dave
« Last Edit: January 25, 2014, 10:35:25 pm by dga »

Offline mmao

  • Jr. Member
  • **
  • Posts: 28
    • View Profile
Re: Open source optimized PTS CPU miner (BETA)
« Reply #95 on: January 25, 2014, 10:50:49 am »
Hm.  What happens if you run with only 1 thread?

What happens if you first run, as root:
   echo 2048 > /proc/sys/vm/nr_hugepages

and then run with only one thread?

If it works with 1 thread, does it also work with 2?

  -Dave

after executing "echo 2048 > /proc/sys/vm/nr_hugepages", it works with 4 threads!!!
and with 4 threads sse4 mode on my i7-920@2.67GHZ, it gives about 170 cpm
it's quite good, thank you!

Offline dga

  • Full Member
  • ***
  • Posts: 122
    • View Profile
Re: Open source optimized PTS CPU miner (BETA)
« Reply #94 on: January 24, 2014, 07:02:20 pm »
Hm.  What happens if you run with only 1 thread?

What happens if you first run, as root:
   echo 2048 > /proc/sys/vm/nr_hugepages

and then run with only one thread?

If it works with 1 thread, does it also work with 2?

  -Dave

I downloaded the beta7.1 static binary, and run it on my centos6 box but failed:

Code: [Select]
using SSE4
spawning 4 worker thread(s)
[WORKER[WORKER[WORKER2] starting
3] starting
1] starting
[WORKER0] starting
Couldn't use the hugepage speed optimization for big table.  Enable huge pages for a slight speed boost.
Couldn't use the hugepage speed optimization for big table.  Enable huge pages for a slight speed boost.
Couldn't use the hugepage speed optimization for big table.  Enable huge pages for a slight speed boost.
Couldn't use the hugepage speed optimization for big table.  Enable huge pages for a slight speed boost.
Couldn't use the hugepage speed optimization for small table.  Enable huge pages for a slight speed boost
Couldn't use the hugepage speed optimization for small table.  Enable huge pages for a slight speed boost
Couldn't use the hugepage speed optimization for small table.  Enable huge pages for a slight speed boost
Couldn't use the hugepage speed optimization for small table.  Enable huge pages for a slight speed boost
[WORKER1] GoGoGo!
[WORKER2] GoGoGo!
[WORKER0] GoGoGo!
[WORKER3] GoGoGo!
connecting to 54.201.26.128:1337
Mining for approx 60 seconds to support further development
Payments to: Pr8cnhz5eDsUegBZD4VZmGDARcKaozWbBc
[MASTER] work received - sharetarget: 03ffffffffffffffffffffffffffffffffffffffffffffffffffffffbeefde4d
ptsminer-dga-beta7.1-avxsse-linux64-static.bin: malloc.c:2369: sysmalloc: Assertion `(old_top == (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t)) < __alignof__ (long double) ? __alignof__ (long double) : 2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t)) < __alignof__ (long double) ? __alignof__ (long double) : 2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed.
aborted (core dumped)

it failed also with sph mode:
Code: [Select]
using SPHLIB
spawning 4 worker thread(s)
[WORKER[WORKER12[WORKER] starting] starting

3] starting
[WORKER0] starting
Couldn't use the hugepage speed optimization for big table.  Enable huge pages for a slight speed boost.
Couldn't use the hugepage speed optimization for big table.  Enable huge pages for a slight speed boost.
Couldn't use the hugepage speed optimization for big table.  Enable huge pages for a slight speed boost.
Couldn't use the hugepage speed optimization for big table.  Enable huge pages for a slight speed boost.
Couldn't use the hugepage speed optimization for small table.  Enable huge pages for a slight speed boost
Couldn't use the hugepage speed optimization for small table.  Enable huge pages for a slight speed boost
Couldn't use the hugepage speed optimization for small table.  Enable huge pages for a slight speed boost
Couldn't use the hugepage speed optimization for small table.  Enable huge pages for a slight speed boost
[WORKER2] GoGoGo!
[WORKER3] GoGoGo!
[WORKER0] GoGoGo!
[WORKER1] GoGoGo!
connecting to 54.201.26.128:1337
Mining for approx 60 seconds to support further development
Payments to: Pr8cnhz5eDsUegBZD4VZmGDARcKaozWbBc
[MASTER] work received - sharetarget: 03ffffffffffffffffffffffffffffffffffffffffffffffffffffffbeefde4d
Segmentation fault (core dumped)

Offline mmao

  • Jr. Member
  • **
  • Posts: 28
    • View Profile
Re: Open source optimized PTS CPU miner (BETA)
« Reply #93 on: January 24, 2014, 10:18:12 am »
I downloaded the beta7.1 static binary, and run it on my centos6 box but failed:

Code: [Select]
using SSE4
spawning 4 worker thread(s)
[WORKER[WORKER[WORKER2] starting
3] starting
1] starting
[WORKER0] starting
Couldn't use the hugepage speed optimization for big table.  Enable huge pages for a slight speed boost.
Couldn't use the hugepage speed optimization for big table.  Enable huge pages for a slight speed boost.
Couldn't use the hugepage speed optimization for big table.  Enable huge pages for a slight speed boost.
Couldn't use the hugepage speed optimization for big table.  Enable huge pages for a slight speed boost.
Couldn't use the hugepage speed optimization for small table.  Enable huge pages for a slight speed boost
Couldn't use the hugepage speed optimization for small table.  Enable huge pages for a slight speed boost
Couldn't use the hugepage speed optimization for small table.  Enable huge pages for a slight speed boost
Couldn't use the hugepage speed optimization for small table.  Enable huge pages for a slight speed boost
[WORKER1] GoGoGo!
[WORKER2] GoGoGo!
[WORKER0] GoGoGo!
[WORKER3] GoGoGo!
connecting to 54.201.26.128:1337
Mining for approx 60 seconds to support further development
Payments to: Pr8cnhz5eDsUegBZD4VZmGDARcKaozWbBc
[MASTER] work received - sharetarget: 03ffffffffffffffffffffffffffffffffffffffffffffffffffffffbeefde4d
ptsminer-dga-beta7.1-avxsse-linux64-static.bin: malloc.c:2369: sysmalloc: Assertion `(old_top == (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t)) < __alignof__ (long double) ? __alignof__ (long double) : 2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t)) < __alignof__ (long double) ? __alignof__ (long double) : 2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed.
aborted (core dumped)

it failed also with sph mode:
Code: [Select]
using SPHLIB
spawning 4 worker thread(s)
[WORKER[WORKER12[WORKER] starting] starting

3] starting
[WORKER0] starting
Couldn't use the hugepage speed optimization for big table.  Enable huge pages for a slight speed boost.
Couldn't use the hugepage speed optimization for big table.  Enable huge pages for a slight speed boost.
Couldn't use the hugepage speed optimization for big table.  Enable huge pages for a slight speed boost.
Couldn't use the hugepage speed optimization for big table.  Enable huge pages for a slight speed boost.
Couldn't use the hugepage speed optimization for small table.  Enable huge pages for a slight speed boost
Couldn't use the hugepage speed optimization for small table.  Enable huge pages for a slight speed boost
Couldn't use the hugepage speed optimization for small table.  Enable huge pages for a slight speed boost
Couldn't use the hugepage speed optimization for small table.  Enable huge pages for a slight speed boost
[WORKER2] GoGoGo!
[WORKER3] GoGoGo!
[WORKER0] GoGoGo!
[WORKER1] GoGoGo!
connecting to 54.201.26.128:1337
Mining for approx 60 seconds to support further development
Payments to: Pr8cnhz5eDsUegBZD4VZmGDARcKaozWbBc
[MASTER] work received - sharetarget: 03ffffffffffffffffffffffffffffffffffffffffffffffffffffffbeefde4d
Segmentation fault (core dumped)

Offline ptsrush

  • Full Member
  • ***
  • Posts: 84
    • View Profile
Re: Open source optimized PTS CPU miner (BETA)
« Reply #92 on: January 24, 2014, 03:01:26 am »
hi dga,

I found that yam support beeeeer.org and report about 7% RJ too.

Since both miner report almost same reject rate, I guess your code is OK and
 don't worry about RJ any more.

Thank you very much.

Offline jernau

  • Full Member
  • ***
  • Posts: 78
    • View Profile
Re: Open source optimized PTS CPU miner (BETA)
« Reply #91 on: January 23, 2014, 06:51:22 am »
You can use just sse4.
PTS: PgiEykg2RATYwWYhFtyNRqwSxQyEApLSmW

Offline Brekyrself

  • Hero Member
  • *****
  • Posts: 512
    • View Profile
Re: Open source optimized PTS CPU miner (BETA)
« Reply #90 on: January 23, 2014, 02:59:33 am »
Just to clarify the miner, does it require AVX or can it be run on SSE alone?

Offline dga

  • Full Member
  • ***
  • Posts: 122
    • View Profile
Re: Open source optimized PTS CPU miner (BETA)
« Reply #89 on: January 22, 2014, 08:40:21 pm »
using same Internet connecion, my ivybridge (avx sse beta7) RJ is higher than haswell (avx2 beta7).

Now on several machine, VL 1013 and avxsse 14.3% : avx2 0.1%.

Ok, replying to a few of these:
 - I've put beta 7.1 online just for avxsse.  I've let a few small tweaks from what will be beta8 slip in, but it's basically the same as beta7 from a performance perspective.
 Changes:
    - It should improve the reject rate.  It's a bit more aggressive about checking for updates now without slowing down mining. (ptsrush)
    - Better handling of and diagnostic messages for out-of-memory / allocation errors. (dclark)

Update:  After about an hour of testing on a 64 core AMD machine:

    789.7 c/m | 12.3 sh/m | VL: 623 (99.7%), RJ: 2 (0.3%), ST: 0 (0.0%)

Looks like this one successfully pulls the submitted rejects down for avxsse also, though an hour isn't quite long enough to say what the overall reject rate will be.

Update 2:  After a day (the c/m and sh/m got reset but VL/RJ didn't):
    758.6 c/m | 12.0 sh/m | VL: 16570 (98.6%), RJ: 228 (1.4%), ST: 0 (0.0%)

Looks solid on rejects.

RJ is reject, ST is stale, VL is valid. 

I'm pondering the open-sourceness.  In the case of GPU, I'm happy - Invictus paid for the release.  Now that I'm doing unpaid improvements to the CPU miner, I want to see how it plays out - but I'm increasingly leaning towards keeping at least some of the cutting edge private as a way to get a bit of return on development time, and trying to keep the open source version updated at a reasonable level that lags a bit behind the latest and greatest but is still a good basis for people who want to learn / explore / improve.  It's a tough question.
« Last Edit: January 23, 2014, 07:39:18 pm by dga »

Offline ptsrush

  • Full Member
  • ***
  • Posts: 84
    • View Profile
Re: Open source optimized PTS CPU miner (BETA)
« Reply #88 on: January 21, 2014, 01:51:51 am »
using same Internet connecion, my ivybridge (avx sse beta7) RJ is higher than haswell (avx2 beta7).

Now on several machine, VL 1013 and avxsse 14.3% : avx2 0.1%.
« Last Edit: January 21, 2014, 03:16:22 am by ptsrush »

Offline ptsrush

  • Full Member
  • ***
  • Posts: 84
    • View Profile
Re: Open source optimized PTS CPU miner (BETA)
« Reply #87 on: January 21, 2014, 01:29:32 am »
Hi dga,

More stats report.

I run your avx2 beta7 for 2 days.

In the beginning, the RJ is almost 0.0%. But time passed, it raise to 6.4% slowly.

I see "too many rejecets (3) in a row, forcing reconnect" several times.

Flowing that error message, the ptsminer reconnect and even more "[MASTER] submitted share -> REJECET"

I had to Ctrl-C to restart the ptsminer.

Thank you.