Most database engines of moderate sophistication straddle the "systems" and "application" line because they start directly managing system resources at a low level. Application servers are a bit more like "applications".
Database engines highlight some of the reasons that Java is a poor systems language relative to C/C++. High-performance database engines these days are usually bottlenecked by memory I/O performance and efficiency, or storage I/O if you under-provision the machine resources for the workload. This is why you see I/O optimizations applied to in-memory databases, for example. Java may be fast at many things, but it is much slower than C/C++ for codes that are bound by memory performance and efficiency. You have to start doing very awkward things in Java to even come close to what comes naturally in languages that explicitly manage memory behavior and structure. As someone who has written a lot of database engine-y code in both C++ and Java, the difference in absolute performance for nominally equivalent code is not small, and is generally easier to achieve in C++. So for low-level performance-oriented systems, there is a pretty strong bias toward C++, particularly now that there is a lot of experience trying to implement the same systems in Java.
For performance sensitive codes, systems programming is really about carefully managing resources to optimize for characteristics of the system. C makes this very easy because it exposes all of it. To the extent that CPU architectures are increasingly bound by memory performance, I would not be surprised to see C++ supplanting Java for certain purposes.
This makes sense. Java was designed more as an application language that works well for codes that are unlikely to make heavy use of the memory system. Its popularity and generality has caused it to be widely used but that does not always make it a good choice outside of its original design case (see also: Perl).
It's mostly a semantics thing, but no: most "systems programmers" would not categorize those applications as "systems tasks". They're just apps. By convention, "systems programming" means dealing with the bottom of the abstraction stack. Java code can't make syscalls. Calling conventions for Java methods aren't specified at the level of CPU registers or instructions.
So writing a JVM is systems programming. Writing a Java-based data store is not, even though other stuff then sits on top of the store.
It is certainly not possible to do syscalls in "Java". A system call is a special CPU instruction (e.g. INT, SYSENTER) not accessible to or defined in the Java VM specification.
What you do to effect a syscall is to call a JNI function to do the work. JNI is a C (!) API, defined in terms of the C (!) ABI for the platform.
And sure: you can generate native machine code in Java, just as you can in python or bash or even BASIC. But you can't call it.
>Unless if people categorized the above software as non-system-programming.
Probably the case, networked services represent an area where performance is at least somewhat a concern, but BitC is trying to address situations where one is writing a memory allocator, not building a server on top of a GC'd language known for having sloppy memory usage.
HBase, Hadoop, Cassandra, GWT tools, MQ, App Servers (Jetty, Tomcat, GlassFish, etc), EhCache.
Unless if people categorized the above software as non-system-programming.