So I’m still doubling around with that odd server issue. I’m getting closer, I know the symptoms, just not sure about the best solution. While doing other things tomorrow, I think I need to bounce it, stator it up, and once it’s started throw one file at it while logging the output of netstat every 30 or so. I’m assuming I’m hitting the built in cap on open network ports with the large number of small files I poke at. Now JobFlow seems to limit 5 files being processed at once just for this reason, but for some reason, I’m still running afoul of ports cycling too fast. Of course, this is all conjecture, so we’ll see if I see the cursed number hit.
Assuming that my auditions are correct, which is often the case,odo I increase the connection limit, add a delay for every file processed, add a delay based on the number of connections used, or something else. Totes not meh job description.