Will this frame hold up?

If your video did not identify any cracks. I could not see any then that frame is good to go. What is your intended use and your skill level? 

I think that it will be fine.

Well, my first thought at the OPs saying it was a Hard Enduro bike being re-purposed, was Why The F**k did you Not run a skid plate?

But, at the end , he said he did. 

What sort, to let such gouging of the frame happen? 

I run Boulder Busters of my own make to Not have such damage  - But yet / yes, you can Still Ding a cradle tube with big enough hits. My old Beast has a couple of (very minor) dings after near on a 1/4 of a Century of clobbering things, even with my Boulder Busters fitted.

Watch the frame like a Hawk, even when Much of the strength from the swingarm pivot area / plates to the downtube Is provided by the rear engine / swingarm interface, And, the lower forward  and  front engine mounts.

 I very much doubt the OP will have a problem. 

However, Observation and Caution are key things to go by, with this.

You can take a grinder and flap wheel and smooth out all the rough stuff which COULD but likely won’t develop stress cracks over time.  Besides that nothing to worry about really.