I just noticed this on our index page, where it lists the members on-line
I wonder why so much traffic?

__________________
At night the stars put on a show for free (Carole King)

Last time, it was because of a very heavy dose of spam engine / spam spider. Hopefully it's a legit record this time.

__________________
[Dr. Horrible]___________________________[Penny]
Listen close to everybody's heart________And you believe there's good in everybody's heart
And hear that breaking sound_____________Keep it safe and sound
Hopes and dreams are shattering apart____With hope you can do your part
And crashing to the ground_______________To turn a life around

A few minutes ago I looked and saw 520 robots, and about 300 other less-robotic guests and the usual many dozen registered members, for a total in the high-800s.

__________________
0 1 1 0 1 0 0 1 1 0 0 1 0 1 1 0 1 0 0 1 0 1 1 0 0 1 1 0 1 0 0 1 1 0 0 1 0 1 1 0 0 1 1 0 1 0 0 1 0 1 1 0 1 0 0 1 1 0 0 1 0 1 1 0....

I wasn't even able to get into BAUT, BA, or UT all weekend on my admittedly very slow dial-up at the vacation home. Is this why?

__________________
Cum catapultae proscribeantur tum soli proscripti catapultas habeant.

Probably not, Treb. It's more likely to have been some sort of unrelated network problem nearer your ISP that segmented the network between you and BA. Switches unexpectedly dying over the weekend aren't all that unheard of. (Meaning the affected site might have been unable to replace it until Monday unless one had a spare immediately to hand.)

But I can't rule it out entirely. It's just that many more folks would have experienced it if it was the bots dragging down the board.

__________________
[Dr. Horrible]___________________________[Penny]
Listen close to everybody's heart________And you believe there's good in everybody's heart
And hear that breaking sound_____________Keep it safe and sound
Hopes and dreams are shattering apart____With hope you can do your part
And crashing to the ground_______________To turn a life around

How are you able to distinguish robots from others?

Why would there be 300 robots on at once? I would think there would
be only one from each search engine at any one time. Do individuals
have their own private search engines, so it isn't just a small handful
of big ones like Google and Yahoo?

What is a 'spam engine'? What is a 'spam spider'?

I know those are appropriate questions to look up at Wikipedia, but
my current Windows/IE install is broken in some way which causes
both IE and Windows to lock up every time I download a page from
Wikipedia. I believe that Wikipedia requests the User Profile from my
computer (the one that Outlook uses), but can't find some file.
The BAUT Members List does the same thing.

-- Jeff, in Minneapolis

__________________
http://www.FreeMars.org/jeff/

"The other planets?
Well, they just happen to be there, but the point of rockets is to explore them!"
-- Kai Yeves

You'll have to ask 01101001. I'm not sure how he determined that. I believe him, though. It's not the first time we've suffered a large spike of them.

The difference is that they're spam robots. Google and Yahoo's spiders are pretty tame. They'll go slow and they'll honor server instructions. Spam robots do neither.

And the "advantage" is that if they hit a site hard enough, fast enough, and with enough individual machines, the admin simply couldn't respond quickly enough to block 'em before they've done what they're here to do.

Sort of. See below.

A spam engine tries to inject spam onto a site or board. The moderation queue catches the more blatant spam posts, and we board members tend to spot the subtle ones that avoid links (usually trying to covertly test a spam vector prior to the actual spam run.)

A spam spider tries to read the whole board and look for email addresses, vulnerabilities, and/or links to other perhaps more vulnerable boards.

Spammers now build or rent "zombie networks" which are basically a large battery of compromised machines. If you have enough of 'em, each machine might only covertly send 20-30 email spams then stop, or they might each run one lightweight spider.

It means that for a site like this one, an attack by 300+ spiders might require blocking 300+ individual IPs, and there's no special reason why one or more of those 300+ can't be hiding on a member's own machine.

__________________
[Dr. Horrible]___________________________[Penny]
Listen close to everybody's heart________And you believe there's good in everybody's heart
And hear that breaking sound_____________Keep it safe and sound
Hopes and dreams are shattering apart____With hope you can do your part
And crashing to the ground_______________To turn a life around

Another part of the high numbers might be that Fraser upped the timeout for inactivity from 15 minutes to about an hour, which basically means users are counted as on for that long even of they only fetched one page.

The Yahoo spider is highly distributed and looks like it's responsible for about 250-300 of the guests we have.

__________________
"God bless thee, my son; I will give thee the greatest jewel I have ...
"The end of our foundation is the knowledge of causes, and secret motions of things; and the enlarging of the bounds of human empire, to the effecting of all things possible."Trying to make sense of computers, The Error Log.

Really? I thought they had a much lighter footprint than that. How unfriendly of them.

__________________
[Dr. Horrible]___________________________[Penny]
Listen close to everybody's heart________And you believe there's good in everybody's heart
And hear that breaking sound_____________Keep it safe and sound
Hopes and dreams are shattering apart____With hope you can do your part
And crashing to the ground_______________To turn a life around

It's hidden well.

Front page :: click Currently Active Users

At the bottom, menu default is: Display: All. Select Search Bots, click Display.

The guest count (e.g. 0 members and 336 guests) at top of list is the count of robots (and the guest count will likely be different if you select to display Guests instead of Search Bots).

__________________
0 1 1 0 1 0 0 1 1 0 0 1 0 1 1 0 1 0 0 1 0 1 1 0 0 1 1 0 1 0 0 1 1 0 0 1 0 1 1 0 0 1 1 0 1 0 0 1 0 1 1 0 1 0 0 1 1 0 0 1 0 1 1 0....

That is only ~300 spiderbot's each having fetched at least one page in the last hour, that's only one page every 5 seconds.
Not quite so unfriendly after all.

__________________
"God bless thee, my son; I will give thee the greatest jewel I have ...
"The end of our foundation is the knowledge of causes, and secret motions of things; and the enlarging of the bounds of human empire, to the effecting of all things possible."Trying to make sense of computers, The Error Log.

"At least" being the key word, Henrik. That's only the lower bound. Not necessarily the upper.

__________________
[Dr. Horrible]___________________________[Penny]
Listen close to everybody's heart________And you believe there's good in everybody's heart
And hear that breaking sound_____________Keep it safe and sound
Hopes and dreams are shattering apart____With hope you can do your part
And crashing to the ground_______________To turn a life around

On another board that I moderate, we've been heavily hit by spambots in the past few days... to the tune of cleaning up 50-60 every few hours. We've blocked a number of the IPs, but quite a few appear to be trojans placed on innocent folks' computers (clicking on a website or opening some emails will do that.)

I should have mentioned that their last activity seemed pretty much spread out over the entire hour, which does seem to indicate they are indeed working near the lower bound.
Only access to the server log will give the actual answer.

__________________
"God bless thee, my son; I will give thee the greatest jewel I have ...
"The end of our foundation is the knowledge of causes, and secret motions of things; and the enlarging of the bounds of human empire, to the effecting of all things possible."Trying to make sense of computers, The Error Log.

I saw about the same, doing frequent requests to list-by-last-activity (or whatever the option is). THe Yahoo spiders all together did about 5-10 requests per minute. The Google bot(s) was small in number but banged harder -- but probably to about the same effect.

__________________
0 1 1 0 1 0 0 1 1 0 0 1 0 1 1 0 1 0 0 1 0 1 1 0 0 1 1 0 1 0 0 1 1 0 0 1 0 1 1 0 0 1 1 0 1 0 0 1 0 1 1 0 1 0 0 1 1 0 0 1 0 1 1 0....

Stupidity is not innocence, since they are so stupid as not to protect themselves online, I say ban all the IPs regaurdless.

Henrik has the right answer. By upping the timeout, it changes the number of people who are currently accessing the forum. Instead of just being people who came by in the last 15 minutes, it's anyone who came here in the last hour. But, this will stop people from having that timeout when they're composing that great big post.

__________________
Fraser Cain
Publisher
Universe Today - Free space news delivered by email every weekday.

THANK YOU!

__________________
"The beauty of that discussion of averages is that you don't have to be an expert in Apollo or in photography in order to see where this time study "analysis" breaks down. You just have to be, well...not an idiot." -JayUtah

Does that apply only to those who have not checked the "Remember me"
box, and thus do not have the cookie set? I frequently take a long time
to compose posts -- especially if I have to do any kind of research --
but don't generally have a problem. ...Although just a few weeks ago
my ISP started, for the first time ever, disconnecting me after 20 minutes
of inactivity. So sometimes now when I click the Submit button, I find
that the modem has to do its BEEP-BOOPs