c# - Creating large amount of tasks/threads and waiting for them all to complete -

i'm writing simple raytracer , i've run runtime limitations because program single-threaded. result i've been finding through google answer type of question 2 or 3 tasks handled.

class program {     static void main(string[] args)     {         var tasklist = new list<task>();          tasklist.add(task.factory.startnew(() => dostuff()));         tasklist.add(task.factory.startnew(() => dostuff()));         tasklist.add(task.factory.startnew(() => dostuff()));          task.waitall(tasklist);          console.writeline("all threads complete");     }      static void dostuff()     {         //do stuff here     } } 

i'm looking @ atleast 10,000 individual threads, if implemented naively. solution above doesn't seem optimal 1 in scenario. there part of standard library supports this, or there nuget package has implemented? might me baing stupid, , >10,000 threads in list not problem @ all. issue becomes when cutoff is. need 12500000 tasks/threads in instances, i'm pretty sure way many list.

below how create new thread/task stands now.

for (var x = 0; x < image.width; x++) {     (var y = 0; y < image.height; y++) {         var coordinates = new vector3(x, y, 0);         var task = new task(() => {             rendersinglepixel(coordinates);         });     } } 

if have list (or other ienumerable<t>) of values want process using multiple threads, can use .asparallel() so.

this intelligently limits number of threads spawned simultaneously, depending on processor capabilities. however, note should use when amount of work per item relatively large.

here's example:

using system; using system.linq; using system.threading;  namespace demo {     class program     {         static void main()         {             var numberstoprocess = enumerable.range(1, 1000);              numberstoprocess.asparallel().forall(dostuff);         }          static void dostuff(int value)         {             console.writeline("thread {0} processing {1}", thread.currentthread.managedthreadid, value);             thread.sleep(250); // simulate compute-bound task.         }     } } 

an alternate approach create task each method call, becomes more difficult know when threads have completed unless store tasks in order wait them complete (but thread pool usage ensure number of threads doesn't large):

using system; using system.linq; using system.threading; using system.threading.tasks;  namespace demo {     class program     {         static void main()         {             var numberstoprocess = enumerable.range(1, 1000);              foreach (int number in numberstoprocess)             {                 int n = number;                 task.run(() => dostuff(n));             }              console.readline();         }          static void dostuff(int value)         {             console.writeline("thread {0} processing {1}", thread.currentthread.managedthreadid, value);             thread.sleep(250); // simulate compute-bound task.         }     } } 

note approach run risk of runaway number of threads being created, if each call todostuff() takes long time. if change thread.sleep(250) thread.sleep(100000) , run program, you'll see large number of threads created.

but best bet use the dataflow tpl.