I found the answer myself on github. Here are the neccesary commands:
git filter-branch --force --index-filter 'git rm --cached --ignore-unmatch data/*' --prune-empty --tag-name-filter cat -- --all
rm -rf .git/refs/original/
git reflog expire --expire=now --all
git gc --prune=now
git gc --aggressive --prune=now