On the quality of recent boxes...

@MrAgent said:
Best.TL;DR.EVER… EVER!

Pretty much. This thread quickly derailed despite what @egre55 said, and I played no small part in that. This bickering is completely unproductive and I apologize to everyone.

I have some input regarding the graph that’s being implemented that I’d like everyone to weigh in on. @ippsec brought this up to me in a PM.

First, the term “CTF” isn’t very telling. Everyone has different definitions of what it means and, at the end of the day, rating a box “CTF” versus “non-CTF” doesn’t do much to inform people. Additionally, there are some CTF-style boxes that are actually very good and have a lot that you can learn from them, such as Mischief. I thought Olympus was a good example of this as well. These are, of course, subjective opinions, but I think what most people mean by “CTF” is not “unrealistic”, but instead boxes that include lots of useless busywork and nonsensical puzzles that don’t teach you anything that you can reapply to real world situations. This in comparison to CTF-style boxes like Mischief that are capable of teaching you tons of stuff. So I think we need to start using more accurate language.

I’d like opinions on this.

Second, I’m going to quote @ippsec directly (I hope you don’t mind) since there’s no need to rephrase things when he already stated it perfectly. This is regarding the graph that’s being implemented:

@ippsec said:
I think it would be better in a less subjective form such as “Binary/CVE/Custom/Enumeration/Network”, and let people decide if its CTF based upon that criteria. If the box has High Enumeration and Custom, chances are it’s going to have some of the elements you don’t like. The benefit with the lesser subjective route is its easier for us (mods) to get right when grading it and it allows for creating a scorecard so people can see if there are area’s they are weak in.

This as opposed to using terms such as “CTF” and “realistic”, when CTF-style boxes can be just fine as long as they teach you things and don’t include ridiculous puzzles in esoteric languages, and “realistic” boxes can be just as frustrating and uneducational as the worst CTF boxes if all you’re doing is crawling through files in a super locked down environment.

I’d like opinions on this as well.

^ @3mrgnc3 and @ch4p

Why don’t you try out a couple of the new graphical rating systems with a test set of users and boxes they have done, and see if they have some agreement on the ratings. I think you’d ideally like people to agree on the ratings but vary widely on whether they liked the box. You wouldn’t achieve the ideal of course but it might help refine the rating system.

Also think about keeping the number of dimensions down or you risk non-responses or random (uninformative) responses. IMO 5 is pushing it.

Since you asked for opinions.

@opt1kz said:
This as opposed to using terms such as “CTF” and “realistic”, when CTF-style boxes can be just fine as long as they teach you things and don’t include ridiculous puzzles in esoteric languages, and “realistic” boxes can be just as frustrating and uneducational as the worst CTF boxes if all you’re doing is crawling through files in a super locked down environment.

I will admit that I haven’t read every post in this thread. I’ve probably read half of them and sorry if I’m re-stating something which someone else has said more clearly, but I think that for me the section of opt1kz’ last post that I quoted above is really what seems most important to me.

That said, I assume that the unfortunate reality is that different people value learning different things. For me, I’m really not all that interested in having to try to puzzle out which esolang some random bit of information lying around on a website most closely resembles and having to script something to convert it into that esolang so that i can then decode it. There are probably some skills to be picked up there, but for me personally it’s hard to imagine where I’m learning anything that useful for anything other than the next CTF by doing that.

On the other hand, although I initially reacted to the python section of chaos by thinking “that’s really unrealistic”, I have to admit that it feels to me as if there are an awful lot of people on this platform who aren’t very familiar with python and if they put the time in to figure out how to write the code themselves, there’s a decent chance that they might learn something which would be useful in a lot of other scenarios. Whether they see it that way or not, though, who knows.

I like the idea of categories and I’ll probably still do boxes even if they have “transform my favorite esolang” sections because even in the case of that box the root section was worth doing. What I’d love to see is some reasonably accurate data about what types of things people can expect to learn from boxes and that being intersected with like/dislike or whatever during the review process of new boxes. If it turns out that every box where you have to write code to decrypt something for instance were to end up with say a 33% dislike ratio, then that probably means that there are a significant amount of people who just aren’t motivated to come here by that sort of thing and as such hopefully the people reviewing boxes will steer the platform towards other things.

@opt1kz @LegendarySpork @deviate @ch4p @ippsec

I pretty much agree with everything stated. With one small addition also.

I would love to see a feedback questionnaire with a freetext section only for box creators to view for their boxes. It’s hard to track feedback in individual DM’s and forum comments and get a good feel for what the whole community’s view on it is. I think the whole pro/lame number thing is way too divisive and a poor way to know what should be done better next time. It’s also highly likely it deters repeat creators if they feel disheartend by not getting the amount of +1’s they had hoped for (maybe sometimes a good thing. Idk?). Luckily I’m a tenacious old ■■■■■■ and wont cry off in the corner if you tell me my kids have cross-eyes and buck-teeth. Just please say it politely ?.

My initial suggestion for the radar graph was mainly to help catagorise what a box is trying to be, versus what the community sees it as. Then a creator knows how well they pitched it.
The identifying catagories can be debated and @opt1kz suggestions sound fine to me.
I think the HTB devs are going to be continuing to code it after the break for the holidays. Maybe @ch4p can weigh in on that one?

It’s clear every one of us continuing this discussion care very deeply about the value of the HTB platform.

Together, imho we can all make it even better than the awesome platform it already is.

I remember before it existed, and how excited i felt for days after i discovered it. I’d been using CTF365 then. I won’t trash talk that platform. It had its small chance to be HTB but didn’t make it.

HTB is simply awesome, and all the guys and girls who run it are all awesome too.

Let’s help them out as a community.

A couple more thoughts.

1 - I’m relatively new here. 78 days, I guess. I learn new things here all the time. Even though I’m not a huge fan, I didn’t even know what esolang was until I came here. Thanks to everyone who’s volunteering time to make this place interesting!

2 - Along the lines of what 3mrgnc3 was suggesting, I think that it is important to understand why someone is thumbs downing something. Whether that’s free form (which may be less valuable from the data science mindset) or whether it’s an ability to select from a list of categories. Personally, in the short time that I’ve been here, I don’t think that there’s ever been a box where I didn’t feel like I learned something useful, but there are certain things which I’m just not that enthused by at this point and I would like to have a way to make that more obvious to people who are volunteering time to make machines.

@3mrgnc3 this is correct, we will be pushing it to production on next update, probably early Jan.

imo something that helps people with limited time decide whether a box has something that they stand a chance of actually seeing on a pentest, or is something that would help on a future CTF is a good thing.

CTF/Realistic is arguably too blunt given the different interpretations of what CTF means, but I’m not sure “Binary/CVE/Custom/Enumeration/Network” does it justice either - there are a lot of real scenarios that involve enumeration for example, which people might see as a yardstick for “CTF” anyway.

It depends what people want categorisation for I guess, if it is to determine realism, than maybe “realism” is an acceptable term?

Hacking communities aren’t what they use to be.

@delo said:
Hacking communities aren’t what they use to be.

Probably not. Can you elaborate a little on that viewpoint?

I mean, for better or for worse?

Sorry for opening this thread back up, but, I wanted to add, as a box creator, I’d love to know why the 42 people who marked my submission as lame thought it was lame (though that’s only 4.2% lame, so at least the box get’s to live on :wink: ). It’d be really helpful. Not sure if there’s a way to do that, but perhaps if you click lame, a open text box that would be shared back to the creator somehow?

Totally on-board with @ippsec about the term CTF. A lot of times when trying to simulate real world stuff you have to make unrealistic choices to do that.

Hello everyone , times ago I’ve submitted a box there and it got rejected the same day , it gove me a bit a incomprehension feeling but after all its the HTB staff decision and I respect that , the free labs are most of time build by community . I know some boxes may be too ctfy but it’s depends by the vision we have … We haven’t all the same experience nor we do pentesting in the same locations … so for example the creds in plaintext on the webserver may seem realistic for a pentester living on a " less " devloped country and unrealistic for a pentester living on a "well " devloped country . In my mind , all depend from the vision of pentesting we have …
If you want practice on a realistic lab I would advice you to try Rastalabs or Offshore :dizzy:
Cheers !!

@opt1kz said:
Edit: Fair warning, there is profanity ahead. …SNIP…

Dude, your whole post I 100% agree and is why I cancelled my VIP and seeking other ‘educational’ platforms.

what are some themes you might want to see covered in boxes. As far as realism, would you like total realism, or just real enough that you have to learn something new?

@Phr33fall said:

@opt1kz said:
Edit: Fair warning, there is profanity ahead. …SNIP…

Dude, your whole post I 100% agree and is why I cancelled my VIP and seeking other ‘educational’ platforms.

It that the only part you read?
Do please share which other training/educational platforms you found.

Thanks.

Just to add my two cents:

Why don’t we have a separate thread to judge the CTF-ishness of the box that is going to be released every week. Soon after the people completed that they can come and post in the thread if they wish.

So in future if anyone wants to work on the box and they don’t like to work on the CTF boxes they can search the thread and decide. As the time passes it may be tough to look back. But still this forum search is good to show the results with proper term.

This may be temporary solution. Hope there again it won’t be a long discussion.

If we can create a kind of Voting we can keep that as well in this forum or external site for each box. It’s easy for users/moderators.

If people don’t bother about CTFish they will ignore the thread and work as usual.

@Tepidangler said:
what are some themes you might want to see covered in boxes. As far as realism, would you like total realism, or just real enough that you have to learn something new?

Hi @Tepidangler. Thank you for asking the question. My preference align with the majority of the other members - Boxes that are both realistic and relevant.

@3mrgnc3 said:
It that the only part you read?
Hi @3mrgnc3 . No, it wasn’t.
@3mrgnc3 said:
Do please share which other training/educational platforms you found.
I didn’t share other preferences as it’s not really relevant. HTB has taken a bit of a setback - It wouldn’t be proper for me to promote other platforms. I very much hope that HTB take heed of all suggestions on this post and return it to the once MEGA platform it was.

While I generally agree overall with the sentiment expressed by the creator of this thread, I have spent some time ruminating about the CTF-y boxes I’ve encountered on the HTB platform. Attempting to find a silver lining to my “try harder” and “just enumerate more bro lol” struggles, I have come to the conclusion that the CTF-y boxes (for better or worse) what they produce in frustration and rage they also produce a corresponding amount of required social interaction on the platform - think forum posting and trading clues on various boxes in the current mix. So it is important to consider the social aspects of CTF boxes and what they provide to the HTB platform.

One sentiment brought out by this discussion that I think is worth it to consider moving forward is that CTF style challenges can be very off-putting to both beginners trying to learn basic techniques to build confidence and seasoned veterans trying to practice their skills so as to not get rusty. Currently, there is no way of telling how a box will be without reading its forum thread. For example, imagine a rating system for boxes that had 4 categories (beginner / intermediate / advanced / expert) - there should be no CTF-y elements in the beginner category, and only VERY sparingly introduced at the intermediate level. Advanced level boxes are where a hacker can be expected to have the base knowledge / google research / social skills required to solve the CTF and know to approach the problem from different angles until the correct answer is identified - along with “earning” the satisfaction of solving it. Expert level is fair game for anything goes IMO - but extreme examples like getting user requires being forced to “guess” reconstruct a one-time pad encryption key for an encrypted archive stegoed into an image of a donkey will be policed by the HTB community (as evidenced by this thread) and the creator will suffer the consequences accordingly.

That being said, I do think an easy-to-use binary choice system like (CTF | REAL) with a percentage total of votes should be implemented alongside the thumbs up | thumbs down after rooting a box. But along with that - I think the ability to vote on the box should be delayed by a day or two after completion so you have time to reflect on your experience and what you learned before providing feedback.

edit : tl;dr there is a balance between Realism and CTF and it must be maintained - have HTB management implement a rating system and reevaluate the state of affairs after an appropriate amount of time.

@Phr33fall said:
I didn’t share other preferences as it’s not really relevant. HTB has taken a bit of a setback - It wouldn’t be proper for me to promote other platforms. I very much hope that HTB take heed of all suggestions on this post and return it to the once MEGA platform it was.

Fair enough. I agree that the mood has changed in the community somewhat.

Hopefully, the new platform improvements the HTB team have planned will please the community and help encourage the kind of box submissions that people want to see.

On a slight side note
There is another problem now with the community slightly departed from this thread (but linked imho)
I think the extent to which so much spoiler info is freely available to people when they hit a wall is another facet of the general disgruntlement felt by many. Especially those that end up going right to reading the forum/reddit/NSF when they are stuck and getting a hint.

it’s like the food addict feeling sick after stuffing all the chocolate cake down their throat then shouting at the catering staff saying they feel sick and hate the food.

I’ve even done it myself previously and so decided to take time off from doing boxes because of that. It was making me feel miserable.
I decided not to bother with my HTB HOF goals, (my employer didn’t care when I got into top 100 :lol: ) but instead to write challenges with interesting stuff on instead as I learned a lot more from it and got a balance of attack/defend skills practice from it.

@3mrgnc3 said:
On a slight side note
There is another problem now with the community slightly departed from this thread (but linked imho)
I think the extent to which so much spoiler info is freely available to people when they hit a wall is another facet of the general disgruntlement felt by many. Especially those that end up going right to reading the forum/reddit/NSF when they are stuck and getting a hint.

I agree this is linked, because I think its the more ctf elements of a box that push people towards looking for spoilers. If a box is simply “follow the methodology” (whatever that might be) then spoilers dont really exist.

I think that people who are faced with things that may be obvious to one person but not another (such as guessing the login to a TomCat portal) are more likely to get frustrated and seek spoilers.

But, saying that, I am not sure it matters. No matter what you do there will be people who want the answer handed to them so they can get more internet points. No amount of cracking down on forum posts or reddit threads will change that but I dont think it matters. Everyone works in their own way and I certainly dont think someone is better or worse than someone else based on what boxes they’ve pwnd on HTB.

@TazWake said:
I agree this is linked, because I think its the more ctf elements of a box that push people towards looking for spoilers. If a box is simply “follow the methodology” (whatever that might be) then spoilers dont really exist.
I think that people who are faced with things that may be obvious to one person but not another (such as guessing the login to a TomCat portal) are more likely to get frustrated and seek spoilers.
But, saying that, I am not sure it matters. No matter what you do there will be people who want the answer handed to them so they can get more internet points. No amount of cracking down on forum posts or reddit threads will change that but I dont think it matters.

If lots of people get used to a workflow that relies heavily on spoiler info, does that then not devalue the achievement of getting higher up in the HoF and the HoF as a whole thing?

I would argue, yes. But I’m very interested in everyone elses thoughts on the subject.