Each subdirectory only counts as one file under that directory. If you have one file per subdirectory you will have the same problem, but if you group files into fewer subdirectories you won't have an issue.
For example:
Dir1
File1
File2
File3
File4
For the below layout, Dir2 and Dir3 will take up directory space of Dir1, but File1-File4 will not. They will take it only from their parent directory (Dir2/Dir3). You can test this with the ls -l command and add files and watch the directory size of the parent.
Dir1
Dir2
File1
File2
Dir3
File3
File4
ls -l
drwxr-xr-x 3 username group 34 Jun 6 09:34 testdir
The 34 is the directory size... add a directory with in test and add files to that subdirectory and you will see the directory size of testdir will not change.
Good luck,
Alex