Clickable errors in Visual Studio Code with a Makefile

I'm trying to work on a Makefile based project with Visual Studio Code. I created a task and now I can run make and see the output including a little bit of coloring inside visual studio code. However, I am not able to click on a compile error and have it take me to the corresponding line of code. I think this has to be possible, and I can't imagine it's hard, so I must be missing something obvious.

I can reproduce it with a tiny example:

main.c

#include <stdio.h>

int main(){
  printf("jo\n");
  unsigned int u = -999;
  int x;
  if ( u < x )
    printf("really small\n");
  asdfasdfa
#error "nononono"
}

Makefile

all:
    gcc -Werror -Wall -Wextra -o main main.c

tasks.json

{
    // See https://go.microsoft.com/fwlink/?LinkId=733558
    // for the documentation about the tasks.json format
    "version": "2.0.0",
    "tasks": [
        {
            "label": "make",
            "type": "shell",
            "command": "make",
            "problemMatcher": [
                "$gcc"
            ],
            "group": {
                "kind": "build",
                "isDefault": true
            }
        }
    ]
}