Questions tagged [parallel-computing]
The parallel-computing tag has no usage guidance.
21
questions
0
votes
0
answers
14
views
Concurrency rate limiter value on a service
I got a C# .NET 8 service. .Net 8 delivers a built in concurrency limiter to limit the concurrent requests made to the service, with an option to define a queue for requests made if their number pass ...
- 97
0
votes
1
answer
54
views
Exploring Advanced CPU Virtualization: Amalgamating Cores for Enhanced Performance [duplicate]
Virtualization experts! I have a question that revolves around a potential CPU virtualization concept illustrated below. Is there an existing implementation of this idea, or are there hardware ...
- 101
0
votes
1
answer
250
views
Setting up slurm with 2 different nodes and 2 different partation on 1 physical server
I have a requirement for setting up slurm on one physical server, with 2 different partation and 2 main node so, need to have,
partation1 wihich need to have node1 and need to be used by group1 user
...
- 35
2
votes
2
answers
420
views
Does the ZFS scrub support parallelization for increased performance, e.g., with a 64-core AMD Threadripper Pro?
I have a 24 drive zpool comprised of 3 RAIDZ1 vdevs running 8 Seagate Exos X18 16TB drives per vdev. This is on a Supermicro MB with a 64-Core (128 thread) AMD Threadripper Pro and 256GB ECC RAM.
...
- 29
0
votes
0
answers
38
views
Optimal Swappiness in Parallel R Processes
I understand that optimal swappiness depends on the application. Database servers often have a swappiness close to zero, while in many other cases it is apparently recommended to keep the default ...
- 103
9
votes
2
answers
2k
views
IBM GPFS : very slow to remove files recursively
To delete files recursively in our IBM GPFS cluster, we use simple unix command like :
rm /my/directories -fr
However deletions are very long to be done.
Problem is that our distributed apps (Spark-...
- 115
9
votes
1
answer
27k
views
How can I tell the maximum threads my server can run?
Here is the machine spec:
CPU(s): 20
Thread(s) per core: 1
Core(s) per socket: 10
Socket(s): 2
Based on what I've read so far, these numbers mean that I can run 20 ...
- 193
0
votes
1
answer
649
views
How to launch a PBS job with hybrid MPI/Openmp
I would like to understand how a GROMACS job launched on my SGI cluster with PBS/Torque using a hybrid parallelization MPI/OpenMPI, works.
The cluster is hyper-threading enabled and each node has 16 ...
- 103
2
votes
1
answer
105
views
R scripts that only run sequential, any way to parallelize its execution to multi cores?
i have some R scripts, which only can run in sequential way, cannot be broken into chunks or any parallel library for R or any other language cannot be used.
Is there any way i can distribute the ...
- 4,319
4
votes
2
answers
13k
views
Scan the full filesystem in parallel with clamscan
I run a clamav scan weekly on my servers. There is one server with a raid6 cluster of 30TB of disk space where the scan take more than 24h to run.
So I wonder how can I run clamscan on the whole ...
- 195
1
vote
1
answer
179
views
Execute script with different argument on AWS instances
I have a script which takes multiple arguments and i need to run this script on multiple instances in parallel on AWS. For example, for sake of simplicity, if i have three instances in AWS, i would ...
- 147
2
votes
1
answer
295
views
Multliple core overload above 100% on Centos7 Supermicro Server
I am running Centos 7 (3.10.0-514.26.2.el7.x86_64) on a supermicro H8QG6 board with 4 AMD 6276 cpus (16 cores), for a total of 64 cores. I use it for scientific computing, and usually everything runs ...
- 51
1
vote
2
answers
170
views
Execute command then parallelize other commands after completion
I'm looking for a single inline command to execute /bin/first and then when it completes execute the following /bin/p1, /bin/p2, /bin/p3 in parallel after.
- 5,518
1
vote
1
answer
71
views
Application caps at 20% utilization per core no matter how many threads are launched
I'm trying to use a Dell Poweredge R900 I got secondhand as a compute pool (4x quadcores gives me 16 cores to run simulations with). It's running windows server 2008 R2 enterprise at the moment.
I'...
- 11
2
votes
3
answers
2k
views
GNU parallel doesn't fully utilize my CPUs
I'm running a command like this on my 36 core server (EC2 c4.8xlarge/Amazon Linux).
find . -type f | parallel -j 36 mycommand
The number of files to process is ~1,000,000, and it takes dozens of ...
- 163