Deep neural networks have been widely used for solving PDEs in recently years. In this talk, we shall discuss some numerical issues for such approaches. In particular, we shall present some recent ideas for dealing with essential boundary conditions, nonlocal operators and effective sampling strategies on unbounded domains.