Directory size is very high instead of 4kB expected size

Issue

A large amount of files was created and then deleted in a directory.
After the deletion, the directory size is very high.
For example:

$ ls -ld .
dr-xr-x---. 7 root root 1114112 Jul  4 13:39 .
                        ^^^^^^^