The desire for “success” is an integral part of the American Dream. College graduates everywhere bask in the glory of their newfound knowledge, in hopes that their education will lead them to be successful in their field, in their lives, and in their homes. We tend to measure success by how much we have, yet that same surface-level measure for success is also at the root of the recent economic crisis. We Americans have a lot, don’t we? And it never seems to be enough. Nice homes, decent salaries, unlimited forms of entertainment at our finger tips. Yet at the end of the day, we can’t even keep the clothes on our backs because we gave every asset we had to get the end result… “stuff”. We sold ourselves short trying to live the Great American Dream. I don’t think this is what the first Americans had in mind when they started dreaming.

We do the same thing to our businesses. We have this picture in our head of what our job or business looks like. We do our best to educate ourselves and acquire the experience necessary to reach success. However, we are selling ourselves short. We want success and we want it now! We don’t have time to properly plan and prioritize… we have a business to run. We don’t value the feedback and abilities of our people… after all “It’s our business. If an employee wants to have an opinion, they can start their own. We didn’t hire them to be talented. We hired them to do a job.”

Does some of this sound familiar? Maybe you’ve heard similar sentiments from your managers. Perhaps this comes across in unspoken communication throughout your organization. Perhaps YOU even think this way but have a hard time admitting it because it sounds so ruthless… so insensitive… so careless. In the quiet corridors of our minds, most of us see the flaws in this way of thinking. Yet, we go throughout our daily routine, getting sucked into this vision of success that has been pounded into our corporate cultures… this idea that “business is business”… that the quickest way there is the best way. As a result of this “do or die” attitude, we never take the time to see the symptoms of our own condition. We recognize that there are some issues that need addressing – the same issues that our competitors deal with – and figure that it’s just a natural part of doing business. After all, high turnover, low morale, shoddy production, employee backbiting and disrespect are normal issues for a growing company, right?

Just like our desire to live the American Dream, we sit in our comfy cubicles and glass offices and find ourselves in a situation that we did not expect. We meant well, but we eventually come to terms with the reality that true success is still as far away as it ever was.

Hopefully, this doesn’t describe you. Perhaps you are more astute than the common professional, or maybe you’ve worked in the trenches and see the reality I’m describing. You may recognize that change is needed, but what?

A common theme among business analysts and authors is that many companies fail due to internal problems, not market changes or supply and demand, etc… They simply implode due to various factors that get out of control over time and lead to lost productivity, poor morale and inferior products and services. These kinds of problems are often highlighted when referring to the differences between two relatively similar companies where one is wildly successful and the other simply fades into the distance or crashes abruptly. After months of taking shortcuts, many companies find that all the capital, marketing and restructuring cannot save them. There is no shortcut to success.

What is needed is a core shift in the mentality behind what makes a company TRULY successful. This requires a drastic change toward more traditional values with the ability to integrate the necessary modern methodologies and tactics that are crucial to survive in a modern world and economy. A shift of this magnitude takes away the “business is business” excuse. It requires much higher levels of mutual respect among coworkers and associates. It allows producers to take pride in their work once again. When the final checklist of accomplishments is being reviewed at the quarterly board meeting, and the items in question are: Did we meet our clients expectations? Did we have positive growth? Did we focus on the right things? Are the employees satisfied and feeling a sense of contribution? Is our turnover lower?… the answer can become be a resounding “YES!”

Whether you are someone’s subordinate or in charge of the whole enchilada, the difference can be made starting today. The crux of the larger problem comes down to how you, and perhaps all the people in your organization, perceive your business… your paradigm. But what about this paradigm is so ineffective? I would imagine that your organization, like most others, puts the majority of its resources into things like: improving the bottom line, meeting deadlines. While these facets of business definitely have a place in reaching and sustaining success, they too often become the only facets that receive any attention. This is due in part to their instant measurability. You always know what your profit looks like. You can quickly determine if you are behind a deadline. We tend to think that what we can easily measure is all there is. And so our progress, success and even failure is measured by these important, but misplaced objectives. Meanwhile, other worthwhile objectives such as longevity, employee growth and client satisfaction get completely ignored. After all, if it can’t be measured and translated into instant profit for the shareholders, then it doesn’t serve a purpose.

This “I want it now!” perspective is common with many other things as well. Various industries and the general population ignore pressing issues of pollution and natural resources, enjoying what the earth has to offer today, without much concern for the future. Essentially, short term demand almost always dominates long term progress and sustainability. It’s a very selfish, impatient and shortsighted perspective.

Other more important factors such as people, culture, future and sustainability are overlooked because they aren’t tangible. They can’t be calculated. They don’t fit nicely on a graph or quarterly report. How do you quantify the quality or satisfaction of your people? How do you measure the nuance of a company culture? How do you determine what your future looks like or whether or not you are moving in a sustainable direction? You can’t really… which is why there will always be organizations whose focus is on nothing but the numbers. Bottom line? Numbers. Products? Numbers. Employees? Numbers. Layoffs? Numbers. Of course, you can do a slew of surveys, create review committees and hire analysts dedicated to the purpose of quantifying the intangible. However, this just feeds back into the bureaucratic measurability of what can be measured and understood. You’ll never have the full picture.

What is needed are people that can, or can be taught to, appreciate the human factors of an organization. People that are willing to take a chance on other people… by letting Bob off a bit early to go take care of his sick kids… by allowing the team to set their own goals, deadlines and expectations (I know. It sounds absolutely ludicrous to have a professional actually know what they are doing.)… in hopes that those people will enjoy their jobs more, feel respected, gain more personal balance and reward the whole of the company with their continued hard work, best efforts and hidden talents. Those with authority can exercise a bit less of it and focus on leadership. They can loosen the reigns a little and allow people to express their talents and interests. They can be willing to encourage an environment of trust, integrity, interpersonal influence and personal growth. People that can pioneer in this direction will be your indicators. They will have the pulse of the organization and will be your measurement. They will have the hunches from which effective decisions can be made.

The kind of choices necessary to successfully and effectively run a business require patience, contemplation and thought for the future impact those choices will have on the organization as a whole. How the business is run, how the projects are planned and how client expectations are set and managed come down to what the decision makers see as important. Long term, authentic, success in business comes down to a focus on priority.

What are your priorities and how do you think your focus affects your success?

fudnik Perspective agile, bottom line, business, priority, profits, success

You know, I’ve had a fun and interesting career thus far. I’ve worked for some great companies and for some duds, but regardless of who I’ve worked for and what I did for them, I’ve always managed to learn a lot. Not just about my work, or the industry, but about the people, the processes, the organization, the obscure or the obvious political structure or agendas, what works and what doesn’t, what motivates and what suffocates, what incubates change and what stifles it. As a matter of fact, I’ve come to recognize in myself that the most fascinating aspects of my past work experience, the most frustrating, the thoughts that keep me up at night, usually have little or nothing to do with my work assignments or specific profession. They are usually about the people, the organizational behavior or the dynamics of a thriving or failing company. There is something in me that is always acutely aware of the dynamic environment that surrounds me.

I’m a tinkerer. I mean, I have friends and associates that tinker with cars, appliances and other devices and they do amazing things. My brother’s talent with bodywork and vehicle enhancements makes Orange County Chopper look like child’s play. While I am also very mechanically adept, and also like to make or work on things of various natures, more than anything else, I’m a tinkerer of the intangible. It’s the things that escape vision or the grasp of the hand that become my mental playground.

What keeps you up at night?

fudnik Perspective

Okay, so I’m working on a new Rails project. Things are coming along great. Then we hit a snag where our SSL is not working as expected. We want it to work on Staging and Production only, and only for the actions that we need them on. So, the SslRequirement Gem did the trick.

However, we have many (and many more to come) sub-subdomains which caused another dilemma. We have a wildcard SSL certificate, however, although we can get one that also handles sub-subdomains, it’s not necessarily supported by the user’s browser. So, our other option was to put all the public stuff on the subdomains and have all the private stuff on a “private”.domain.com address which would adequately be handled by SSL at the application and certificate levels. After some finagling, I managed to dynamically change the subdomain based on whether or not the action requested should be SSL’d.

Everything seemed to be humming along, but this new code snippet was relying on something that we hadn’t previously tested thoroughly… sessions. Session are just supposed to work right? However, evidently they don’t work by default across subdomains. So, after some hunting around, this little snippet put into my “/config/[environment].rb” file did the trick.

Evidently, this tells the session to share across anything within the main domain. You can also restrict it further by using “.[subdomain].[domain].com”.

Works like a charm.

fudnik Ruby https, Rails, SSL

What is click fraud?

Let’s start out by defining some key terms that are important in understanding click fraud.

• Advertiser – The entity that pays money to get traffic to their site in the way of bidding on keywords or topical categories (bid auctions).
• Publisher – Any entity which displays advertiser ads on their web site or in some other publicly viewable medium.
• Visitor – A legitimate user who clicked on something to get to the appropriate target web site.
• Click – A visitor to the advertiser’s site that came by route of one or more publishers.
• PPC (Pay Per Click) – An internet advertising model where the amount the advertiser pays is dictated on a per click basis for the terms (keyword or categories) being bid on.
• CPC (Cost Per Click) – The amount (bid price) paid by the advertiser to receive one visitor for a particular term. The amount is paid only if the visit occurs.
• CPA (Cost Per Action) – An advertising cost associated to a particular desired visitor action, i.e. purchased a product or service, filled out a survey, or signed up for a newsletter.
• Conversion – A completion of the advertiser’s desired action under a CPA advertising model.
• Click Stream – The route the click traffic takes from the time the click is made through the time the web user arrives on the advertiser’s targeted URL. There can often be URL redirects and several publishers (usually tracked by cookies or ID’s in the URL) that receive information for each click, completely transparent to the user.
• Rev Share – A single publisher’s fraction of the revenue generated by specific click and conversion sources. For example, a smaller publisher might arrange to send click traffic into a larger publisher’s click stream, providing the larger publisher with more traffic and retaining a 5% rev share of the total per click amount for the smaller publisher. Rev Share can be seen as a multi-tier sales commission.
• Ad Feed – ad listings/data provided by an n-tier publisher by request to display to users on another publisher’s web site or application.

Click fraud, generally speaking, occurs when something (person, web bot, etc…) posing as a legitimate internet user follows (or clicks) a paid advertisement URL to the advertisers web site from which money is generated for some entity other than the advertiser.

An advertiser pays good money for advertising, expecting that a portion of the traffic received in return will generate revenue in some fashion. Non-legitimate visitors produce bad clicks which in effect spend advertiser dollars with no hope of a return for the advertiser. This expense is instead divvied up between the layers (rev share) of publishers that are likely to be present in the click stream. Publishers, especially the ones on the end of the chain, often have the most to gain from this practice and will devise all sorts of innovative ways to game the system. Larger publishers in the click stream will often ignore or downplay this activity, knowing that it lines their pockets in the process.

In short, advertisers are being hijacked of their advertising dollars from inflated term bidding marketplaces because of traffic that is posing as real, live, interested web site visitors. It is theft akin to diverting fractions of a penny from financial transactions to a private account.

What is a valid visitor?

A non-valid visitor is essentially anything that can pose as a user that has no ability or intention of producing revenue for the advertiser, however, there are endless ways this can happen. A fraudulent click bot (any automated application used for the purpose of methodically sifting through sites in order to perform a specific operation) can target specific sites, following advertiser links and sending traffic header information which falsely identifies the bot as a user from a real browser. These bots can get very creative by spoofing IP’s, using proxies and randomizing the browser agent and other key data elements.

This can get somewhat fuzzy, as there are also legitimate scripts and bots that, although not valid users, are not considered fraudulent. Their purpose is to scan the internet, spidering through sites to map them out, extract data for search engines, and many other reasons. While no fraud is intended, they create traffic that will create “clicks” by following a URL and ultimately charge the advertiser for the traffic, unless some kind of safeguard is put in place to filter this traffic.

A valid visitor is a real person using an actual web browser (or application where ads may appear) that clicks on the ad URLs of their own decision and has an interest in what the advertiser is providing, therefore potentially resulting in a conversion for the advertiser. A valid user is always a person, but a person isn’t always a valid user. Even a real person can be considered a fraudulent user (more on this later).

Location, location, location

Publishers provide a means to create clicks that lead to advertisers’ web sites. There are many ways this can happen.

A publisher can have an existing site which provides something of value to a demographic and therefore gets a lot of “organic” search traffic. The publisher would like to further monetize their website, so they tap into an ad feed, parse the data and place the ads on their website. When an interested user clicks on one of the ad links, a click is registered through one or more publishers resulting in a visitor to the advertiser’s site. The web site publisher then gets paid for the click. This same process can be done for applications which display ads. Banners function in a similar manner.

Many publishers are not simply successful web site owners trying to maximize online real estate. Many will funnel traffic through their system without involving any user-facing applications or web sites. A publisher can get ads from another publisher’s ad feed, determine their own pricing model and associated key words, then take those ads along with others and insert them into Google’s or Yahoo’s ad listing via an API. This is one way in which a publisher becomes an advertiser. They play a publishing/distribution role with their ad providers. However, they play an advertising role with their ad distributors as they are purchase traffic at newly negotiated rates, often based on quantity or key word variations.

Publishers will often create a combination between monetizing online real estate and utilizing ad feeds. They can do this by creating dynamic, topic-based web sites with a search engine or topic directory facade. Sometimes they are hand created with relatively decent designs and basic content. Often they are purely dynamic, totally created on-the-fly based on the user’s search term. The goal of these sites is to do nothing more than gain interest from the user in order to get them to click on an ad link which will bring money to the site owner. These sites are often never ending webs of topic pages where the user never really finds what they are looking for. Fraud is common where the link text does not actually represent the target advertiser URL. This is fraudulent because a user may click on a link that says “pet supplies” when the actual URL takes the user to an advertiser who sells “college text books”. Assuming that “college text books” is a more costly search term, when the advertiser pays for the click, the publisher gets paid the higher price. The user, who still hasn’t found what they are looking for, will usually continue to click on links hoping to find what they are looking for, costing many advertisers money while making the publisher wealthy.

Arbitrage has been a recent problem in the industry. The idea is to sell something back to the ad provider at a higher price than what was originally paid for it. Using this method, an entity can buy click traffic (AdWords) for really inexpensive keywords in a particular category, then direct the traffic to a simple web site that shows nothing but ads (Google’s AdSense) that are in the same or similar category but cost more per click. Nothing of value is provided to the user, except more ads. When a user does click on one of the ads, the entity may get paid $0.25 for an ad that only cost them $0.15. This entity essentially plays the role of both advertiser and publisher and provides virtually no technology or service other then automatically reselling the click traffic. This creates a superfluous, middle-man tier that directly drains money from the original ad provider (Google).

A web of deceit

While a good many publishers are running a perfectly clean business, this picture gets muddied up by the web of relationships and interaction involved in the online advertising sphere. There are often complex relationships involved in bringing a user from their click to the final destination. As previously described, you can see how advertisers can also be publishers and visa versa, creating an interesting matrix of profitability and fuzzy responsibility. For all intents and purposes, most players in the PPC industry can be considered both, including most of the innocent small website owners who not only buy traffic but also show ads on their site. So, for general purposes, consider the relationship diagram below.

If a fraudulent click occurs, and there are 4 publishers paid in the click stream, where do you go for answers? Much of the click-related data has a possibility of being faked and the sheer quantity of it turns making anything useful out of it into a daunting task.
Perhaps you start with the publisher at the start of the click stream where the click happened. Perhaps they are the most likely to falsify the click. Do you collect the money back from all the publishers even though 3 out of 4 were operating legitimately. What if the second publisher in the click stream knows that 80% of their traffic comes from publisher 1, so they setup a click bot that hits specific sites provided by publisher 1. This would cause fraudulent clicks to the first publisher’s traffic that they weren’t even aware of.

The art of war

There are several different methods being used to address the click fraud problem. Some of the larger ad providers (such as Yahoo) provide publishers with a traffic quality score (based on conversions and internal statistics) which determines how much and of what quality traffic they continue to receive. The greater efforts the publisher makes to clean up their traffic, the more likely it is they will stay profitable.

Those that are trying to adequately deal with click fraud will often use a combination of their own internal custom methods with a 3rd party service to help determine which traffic is fraudulent. The methods used will often look at the same data in multiple ways and work in conjunction to derive something useful. Internet traffic is highly irregular and unreliable when trying to determine something accurately, making the challenge a moving target. With this in mind, a great portion of potentially fraudulent traffic cannot be deemed 100% fraudulent, leaving many methods to rely on some sort of fraudulent probability scale employed to determine whether or not the affected clicks generate any income for the publishers.

Server-side and client-side validation

Both the web browser (or other ad display client) and the web server can provide a lot of information about a user and their activities. However, anything useful needs to be derived from a series of time stamps, IP addresses, generic browser information and any ad/click related data that can be gathered. There are many arguments regarding using server-side and client-side data for this type of validation. For the most part, server- side data (database records, web server logs, error logs, headers, etc..) can be considered more reliable than client-side data. Server data can provide much in regards to the click, the ads, user location and environment, but can only be gathered in spurts AFTER actions have been taken, leaving lots of room for assumptions regarding the overall user experience. Client-side code, while often likely to be tampered with, can provide a plethora of data regarding user events, intentions, order of events and environment that is not available to the server. Properly obfuscated and used within a limited scope of reliability, client-side validation can filter out a large percentage of fraudulent traffic, leading a handful of click protection companies to rely on it heavily in their products.

Forensic and symptomatic analyzation

Many click fraud detection systems attempt to focus more on factual information and on tracking/gathering more data from the involved symptoms. While this forensic approach can provide a good idea of the concrete data available, falsified or lacking data still leaves a large margin of error in any analysis. As in any adequate security measure, the more the behavior in question is understood, the better it can be targeted and stopped. With click traffic, several valid behavioral assumptions can be made if certain data points exist. So, while the data may not always be completely reliable or correct, using a more symptomatic approach to analyzation can provide many clear behavioral probabilities, which tracked over short periods of time, can isolate bad traffic sources.

Realtime and post analyzation

Server-side analyzation is currently the most prevalent among custom and third party detection solutions. This is due in part to the consistency and availability of server logs that already track many of the needed data points. However, pure server-side analyzation also does not require front end integration, making 3rd party integration relatively simple. A major downside to this method is that all relevant data is analyzed long after the events have happened which only makes it possible to avoid certain traffic sources in the future and makes it necessary to have a feedback loop which tells the analyzed system to undo charges for clicks hours our days after the events have happened. This not only becomes an accounting nightmare, but also misses the opportunity to catch fraudulent behavior as it’s happening. Client-side data coupled with server-side data in a realtime system can identify patterns and known behavior models as they are happening, making it possible to stop fraud short in its tracks before large sums of money are wasted on fraudulent clicks and before upstream publishers and advertisers are affected with the same bad traffic.

In-stream and out-of-stream validation

If the click fraud detection system is realtime and it is done internally or is fully integrated with a third party system, this is considered in-stream detection. Many publishers, especially the small ones with no or little technical resources, are unable to meet the requirements necessary for a full integration, so an out-of-stream option may be provided by a third party. This requires very little integration and allows the provider to capture data (using some kind of tracker pixel or JavaScript ping back) sometime before the click and again sometime after. This methodology is a hybrid between realtime client-side analyzation and pre and post server-side analyzation but doesn’t not offer all the data points and reliability of a full, in-stream integration.
In a model where clicks are rated on a fraud probability scale, any combination of the above methods can be used, within adequate constraints, to provide insight and restriction on a click’s validity.

Smoke and mirrors

One of the greatest challenges facing click fraud security is the rate of change. Browsers are constantly changing. New plugins and their capabilities are constantly making waves in website development. Developers are constantly finding new ways to leverage these tools to their advantage while fraudulent parties are doing the same. Even applications such as email clients are posing great threats to valid click traffic in ways that were not expected.

Key data points

There are many data available for use in click validation, and they are combined and used in many, many ways. For purposes of this article, the many ways these data can be used and analyzed cannot be covered. Below are some of the more common data points.

Server-side

• IP address. Known bad IP blocks can be ignored and patterns can be determined from recurring IP’s. Geographical location can also be roughly determined.
• Proxy, if used. Proxies can be blocked or at least treated with more caution.
• X Forward For. This is a value that is often present when a proxy is used to show the “actual” IP address.
• Browser Agent. This includes the browser name, version, operating system, etc…
• Referrer, if available, which includes the referring domain. This is the web URL that the click came from. This may be non-existent or easily faked.
• Session data. This identifies a particular user’s set of interactions, if the session data isn’t being purged from the user’s end.
• Cookie data.
• Time of initial impression (page view).
• Time of click.
• Other data specific to the click traffic, i.e. click ID’s, advertiser ID’s, publisher ID’s, impression data, etc…
• Conversion. Did a conversion occur after the click?

Client-side

• JavaScript enabled? Sure, there are potentially valid users out there that don’t allow JavaScript, however, if the user’s agent cannot process JavaScript, then the likelihood of them being able to complete the click process and follow through to a conversion becomes negligible.
• Cookie data. Can it be set and read?
• Mouse interactivity. Helps to validate actual presence of a user, versus a bot. Mouse rollovers can also be tracked an sent to the server in real time to help determine mouse movement patterns (see PTR below). Was there actual mouseover activity on the ad link that registered a click?
• Parent window domain. Useful when ad links may be present inside of an iframe.
• Window size. If the window is too small, it is likely that the publisher is trying to mask links so that the user doesn’t know what they are actually clicking on.

The key to using this data and filtering out bad click traffic is to understand what the fraudulent party is trying to accomplish, identify patterns or oddities, then create methods of validating and deflecting the behavior. While the landscape constantly looks different, there are some methods that fraudulent parties will use that seem pretty consistent.

Methods

Standard bots

Problem: Recognized bots such as the ones run by Google, Yahoo! (and hundreds of other search engines) will create lots of non-user traffic which will often inadvertently create click traffic in the process. This can account for a relatively high percentage of actual site traffic, causing a publisher to make a lot more money than they are entitled to. It is not uncommon for a publisher to implement a click fraud filtering system, only to find out that more than half of their traffic was non-user traffic, effectively slashing their profits. While this traffic isn’t considered fraudulent, it should be filtered out so that advertisers don’t get charged for invalid clicks.

Solution: As a primary filter, don’t allow any known bots that are correctly sending their agent string to register a click. Then put other limits in place regarding the number of clicks allowed by certain IP’s within a time period. Could a cookie be set? Can the requesting agent handle JavaScript? This should catch the bulk of the harmless ones.

Click bots

Problem: One of the primary ways fraudulent parties attempt click fraud is through automated bots. These bots will target certain sites where the target ads are known to be and very efficiently simulate clicks at a very fast rate. New bots are consistently smarter and attempt to simulate user behavior, environment and even run JavaScript.

Solution: Additionally, do some consistency checks for valid browser agents and filter out the blatantly obvious ones. Put in place click restriction limits from a single IP. Even IP’s with large offices behind them should not generate hundreds of clicks on the same ad within a few minutes.

Spoofing IP’s and user agents

Problem: Many click bots and similar systems will take the next obvious step and attempt to make each click look like a unique and valid user by randomizing fake IP’s and incorrect User Agents.

Solution: This is where client-side checking can come into play. If JavaScript is enabled, assume the agent is valid and proceed with mouse event and page/browser property checks. Does a cookie check reveal that the user already clicked this ad recently?

Falsified ad text and diversion

Problem: Ads are displayed with text that is more likely to get the user to click on the link instead of text that accurately describes the ad target. This fools users into clicking on multiple ads that they are not interested in while they look for one that is legit. Often they will be offered links to similar topics that attempt to keep them looking for what they want until they click on an ad. Sometimes, this textual falsification may simply be to show a more common topic in the ad text but have the ad link go to a much higher paid PPC ad. For example, the ad may be for a $0.20 PPC “ring tone download” ad but will go to a $20 “mesothelioma” ad.

Solution: Other than random manual validation of publisher websites, some log analysis can be done on the higher payout PPC traffic to see if any particular referrers have unexpected amounts of traffic to particular keywords and if the referring site content matches the keyword.

Hidden browser layer

Problem: The publishing web developer will blatantly put a transparent layer over the page so that regardless of where a user clicks, they click on a link that takes them to an ad. It is possible to get away with this on a small scale for some time before being caught.

Solution: Because this is a real user, it is up to the ad provider to notice the problem (which can be flagged by lack of conversions from the site referrer) and then go to the site to manually verify the problem.

Hidden clickable iframe

Problem: This is similar to the hidden layer problem, although the web developer puts a transparent 1 pixel by 1 pixel iframe under the mouse pointer which follows the mouse everywhere it goes on the page. Wherever the user clicks, they generate an ad click without knowing it. This can also be tricky to notice and track down.

Solution: This may require some manual checking, but a check for mouseover events on the other ads in the feed may reveal that the user is not able to view the other ads.

Small-scale manual clicking

Problem: Some people looking to make a few quick bucks may simply manually click on the ad links. They may get their friends and associates to do the same, especially for higher priced PPC terms. This is very unsophisticated, but it is easy to stay under the radar until the PPC costs or number of total clicks and conversion ratios becomes noticeable.

Solution: Check for recurring traffic patterns and IP similarities Also check for conversion ratios on curious traffic..

Paid to read (PTR)

Problem: This is a more organized version of the small-scale manual clicking method and often starts as such. In order to keep from being detected, a fraudulent party needs to find ways to make the traffic look like valid and interested users. The users need to have varying IP addresses and need to click on a variety of ads to keep from generating any obvious patterns. Publishers will pay users, many foreign, to do nothing else but click on ads in return for a fraction of the rev share. These users will often click on links from their email clients which won’t send a referrer.

Solution: In this situation the user will show a mouse presence, but may continually click on the same link without “browsing” and rolling over any other links. If the rollover to click ratio is near 1:1, this can be a flag of PTR traffic.

Improper Traffic Purchasing

Problem: Many web site owners will attempt to purchase traffic that is outside the contextual topic of the website intent in order to increase traffic. For example they might purchase traffic for the term “hot rod photos” but the
site topic may be “ring tone downloads”. This unqualified user traffic will often click on some of the displayed ads since they didn’t find what they were looking for.

Solution: The big ad providers will have someone manually verify that a company’s website content matches the terms they are purchasing which stifles much of the problem. Google’s AdSense automatically generates the appropriate ads based off the site content. However, if an ad feed is being given to other publishers, there is not chance for interaction with something like AdSense. Conversion ratios and overall traffic quality can be good measures to raise flags when something looks unusual.

How big is the problem?

Estimations as to how many dollars are wasted through click fraud are all over the place. Realistically, no one really knows for sure. Who gets blamed in the tightly woven web of PPC advertising?

When working with a click fraud detection company, the publishers and advertisers we would deploy would often become very disillusioned with the quality of their traffic. Most of them would go through an initial phase of denial, swearing that our system was broken. We would then go through a phase of traffic validation to substantiate the click fraud claims. After a few weeks of traffic adjustments and analysis, some clients would realize that 60% to 90% of their traffic was completely bogus. Even though they had some incentive to clean up their traffic, they would often drop the click fraud detection service because cleaning it up would mean steep cuts to their revenue. At up to 90% fraudulent traffic, this shows that entire companies are thriving on a bubble of almost pure fraud.

Continuing problems…

Click fraud inflates the PPC market causing continuous challenges for advertisers and the market as a whole. Inflated bidding competition drives term prices up and the likelihood of a conversion down making PPC advertising a questionable long term strategy. But like the demand for oil, PPC advertising costs will continue to soar but won’t likely diminish the PPC market because of advertiser’s dependence on online advertising.

Can click fraud be stopped? A higher standard must be set, however, this is a moving target. Click fraud methods are constantly changing and becoming more complex and those players that are deeply embedded in click fraud are constantly ahead of the game.

Perhaps the question that really needs to be asked is whether or not click fraud be controlled within reasonable limits and allow the PPC market to continue to thrive. Like any other type of security or validation, keeping up with and adjusting for the dynamics of common methods and best practices is a good place to start. Ad providers like Google and Yahoo! need to be more stringent on traffic source quality and create tighter restrictions for those whose traffic is questionable. Safeguarding against click fraud needs to become a standard expectation for operating as a publisher in the PPC market. Publishers need to consider taking a pessimistic approach to their own direct traffic and their applications should reflect this, stopping bad traffic at the root of the problem. Advertisers and upstream publishers should consider taking an optimistic approach to the traffic that passes through them (since they don’t have direct access to the original click context and environment), but still develop the tools necessary to analyze and reject blatantly fraudulent sources.

Do your part!

The best way to get started with validating clicks is to take the first step of filtering out known web bots. Since this accounts for much of the non-convertable traffic which advertisers pay for, it can make a huge dent in the problem. Second, start putting the more advanced validations in place as described above. Third, if necessary, start working with a 3rd Party traffic validation company like Click Forensics, Anchor Intelligence or ValidClick.

fudnik Development bad traffic, click bots, click fraud, click spoofing, CPA, hacker, online advertising, PPC, security

I’m not one of those developers that likes to waste time setting up my dev environment. If I have a project to complete, I should be coding, not messing with config files, compiling Apache or messing with PHP to load that extension I just found out I needed. I like messing with my machine but not with an impending deadline.

With all this in mind, I’ve tried to simplify my entire dev environment over the years… not from a perspective of using simple tools and sticking to basics, but instead from a perspective of optimizing my workflow and keeping my development moving. In the middle of a project, I feel that systems admin focus should be on tweaking the production machines, rather than screwing around with my local dev box.

Always looking for ways to make development easier, I decided to give the Zend Server CE (Community Edition) a try. The idea is that it installs your PHP, Apache, MySQL (with PHPMyAdmin) and a great management console that allows you to install extensions with just a click. You can still customize our Apache conf and other things, but it works well out of the box (I’m on a Mac). While you can run it along side another Apache installation, I tweaked it to run on port 80 and handle multiple virtual hosts. While this may not be ideal for all teams, it can allow everyone to have the same environment without having to mess with poorly updated all-in-one dev environments.

One of the reasons this excites me, being on a Mac, is that every time in the past that I’ve updated my Mac OS, the install kills something on my system causing my development environment to go all wonky. Then I have to spend precious work time to fix it. The Zend Server CE install keeps everything nice and tidy and, to my knowledge, doesn’t rely on other stuff outside of the install to function (unless you are setting host entries in your hosts file, /etc/hosts on a Mac).

This, oh so sweet, environment gets a little better. While I still have a variety of development tools at my disposal, my main IDE has become Zend Studio (Eclipse). I know, I know… there are a lot of purists out there that say it’s too heavy, or sluggish, or isn’t simplistic enough. There are occasional bugs or things that annoy me, but at the end of the day it is integrated enough that it lets me get my work done. That’s the whole point of an IDE. It also integrates with Flex Builder plugin which is a plus for me.

fudnik Development, PHP, Reviews Flex, IDE, performance, PHP development, Zend Eclipse, Zend Server CE, Zend Studio

Agile development methods have really taken the IT world by storm. In the last few years Agile has become THE way to manage and develop software, especially among young, emerging companies. It brings to the table a flexible model for communication and progress as well as a sense of anti-corporatism which is heavily embraced in many IT workplace cultures.

While this almost hippie-ish movement of peace, love and agileness has really relaxed a lot of work cultures and has been a boon for productivity and customer interaction, there are some often ignored pitfalls which eventually leave a work culture devastated and disallusioned.

Do it for the right reasons
It’s not enough to adopt Agile just because it works well for some or because you read about it on a trade blog. Agile, or any other hastily adopted process or methodology, cannot solve all your problems. It will simply make you more of what you already are. Your weaknesses, if not already apparent, will eventually surface and you must be ready and willing to acknowledge and address them.

Successfully adopting any methodology like this requires that you have an adequate paradigm about people, business and clients which instills respect and integrity and is in sync with the methodology. If your efforts are only surface-level rhetoric, and no paradigm shift occurs, the process will fail and you’ll be looking for the next “great thing” to fix your woes.

Use best practices
While Agile lends itself to a more rapid pace of development, it can be easy to leave crucial parts of the SDLC out of the equation in the interest of time. Adequate quality assurance and testing are often the first to go. Test-driven development, which utilizes a testing process as part of your development, is a great way to minimize QA overhead while maintaining work quality. Building test code as, or before you develop may add a little to your initial timeline but will result in fewer deployment panics and provide built-in specifications for your code to adhere to.

Don’t sacrifice quality
Cutting corners is a big no-no. Decide what are features and what are bugs. Determine which of them are in your critical path and develop them properly. If you can’t do them right, choose not to do them or arrange for more time to complete the project. NO ONE benefits from poorly thought out, shoddy work. Management only seems happy until they realize the problem they rushed you to fix ends up worse than before. It is the developer’s job to speak up and communicate risks and issues which then translate into proper timeline and feature negotiation.

Don’t ignore problems. Moderately plan for the future and proactively address problems and improvements through iterations. Ignored problems build up over time and eventually result in a complete rewrite. Iterative development can be your friend. Keep track of issues and slip some into each iteration so you can keep up with the change.

Be realistic
There is an old project management addage that explains how with every project, three factors are desired: speed, low cost, and great quality. You can pick two. Having all three is a fantasy propagated by poor sales teams. This is because any improvement in any one or two of the factors will negatively affect the third. For example, if speed is crucial, it will likely affect quality and cost. If very low cost is required, completion times will often be longer and quality will suffer. The only way to realistically improve one of the factors is to improve your effectiveness in all three of them. Attempting to use Agile development concepts to short-sightedly manipulate any of these factors is counter-productive.

Avoid burn-out
Finally, keep in mind that overworking your developers is counter-intuitive in an Agile model. With a more top-down, waterfall approach, you may get away with piling on extra hours, shoving more into a deadline and driving with a whip. Burn-out doesn’t make for solid code, good morale, communication and low turnover; all which are factors behind a well-functioning Agile machine. Utilize iterations to drive realistic deadlines and continually reassess based on top priorities to keep everyone focused on the same goal.

fudnik Development, Technology agile, project management, QA, SDLC

As many of you know, I love Apple products. Not to the exclusion of the everything else, but I’ve got to hand it to them. They’ve created product lines that, despite the often steep cost, attract users and keep them reeled in with great user experience and phenomenal practicality.

One thing has really been bugging me though… this whole issue about Flash on an iPhone. It doesn’t look like it’s going to happen anytime soon and if it does, it will likely be a trimmed down, “do nothing” version. See the Wired.com article.

While I love my iPhone and a great part of the internet can be seen on it, it really ticks me off that they are essentially censoring my use of the internet. Not because of content appropriateness or service bandwidth issues and restrictions. They are censoring the internet on my device simply so they can keep a tight grip on the money they get from the App Store and the other paid-for downloadables that they provide. If you’re going to give me the web… give it all to me and let me decide what to view.

Sure I am missing a few fun games and other things from the world of Flash. But I am also missing access to some very important websites and data that I would like to get on my iPhone, including Adobe Flex-based reporting interfaces from work.

Apple, if you’re not going to allow Flash on the iPhone, at least give me a legit technical reason or something. Otherwise, find a way to make it work. It seems ridiculous that the “most advanced” phone device in the world, that does SO much, cannot even offer Flash. Meanwhile, other second-rate devices are, simply because they can. iPhone users pay good money for their phone and service (not to mention all the other Apple products they have also likely latched on to). It’s an insult really, that many of us professionals own these fantastic phones that we use for so many facets of our business and personal lives… but we can’t have Flash because Apple wants to take an even bigger slice of our paychecks.

I’m very curious to see how they handle this issue. I’ve loved Apple because of the flexibility, the power, the beauty of it all. But, this issue seems a bit too Micro$oft for my liking.

fudnik Reviews, Technology Adobe, Apple, censorship, flash, Flex, iPhone, ria

I’m never really that surprised when I see the latest gadget on the shelves or on one of the many gadget websites that are for the express purpose of flaunting the fact that you don’t have the latest and greatest “thing”. New things are…. well old. There is always something new. There is always an “upgrade”, “new model” or something “totally revolutionary”.

Revolutionary? That word has been so watered down over the years… it almost lacks meaning altogether. Is shoving 100 gigs of ram onto a USB thumb drive revolutionary? Well, no… cool and complex, but not revolutionary.

Today, I saw something that ACTUALLY impressed me. Not in the “ooh”, “ahhhh”, “gotta have it” kinda way. It caught my attention in the overall purpose and drive behind the innovation. I’m talking about the company TerraCycle, Inc..

Okay, so I’m not typically the tree hugger type, but I consider myself conservation minded. Regardless, this is just cool. TerraCycle, Inc. takes trash, just everyday trash, and makes stuff out of it, reuses bottles and other packaging, etc… What caught my interest is that most other companies continue create new “stuff” that adds to the constant barrage of “buy this” and “get that”, while adding to the mountains of trashed iPods, disc players, spent PC’s and other trendy junk already in the landfills.

While I’m not likely to go out and buy one of their “Capri Sun” handbags or a jug of worm poop, it’s their tactic and business model that I find interesting. What if we applied this to technology? What if instead of making yet another shopping cart, blog system, CMS or code framework, we actually put effort into building on top of or improving what we already have access to? What if we could find TRULY creative ways to reuse the mounds of rejected hardware filling up our landfills for something other than sculptures and doorstops? Sure, there are several developers and companies that do that, but most don’t plain and simple.

I think perhaps there is a lot to be learned by looking at this way of doing business, not only as individuals but as an industry. As developers, can we add value to our industry and the world? Or are we filling the net with endless piles of already done cruft destined to be forgotten about next week for the latest and greatest cruft? Admittedly, every day of our career can’t be overflowing with pure innovation and meaning, but looking at TerraCycle, Inc. has me thinking about how I will spend my energy and resources in the future.

See a video about TerraCycle at CNN.

fudnik Reviews, Technology recycle, reusability, Technology, TerraCycle

Okay, well not quite, but I thought this was pretty amazing. GE “brings good things to life”, almost literally.

For those of you that have seen Minority Report, you know that that people have been trying to recreate that type of computing model in the real world since the movie came out. Well, it doesn’t quite exist yet, but GE may be headed in the right direction.

As a way to draw interest to their Smart Grid energy technology, GE has created an interactive 3D experience that is pretty startling at first. It almost seems unreal… until you realize that it is actually interacting with you. Check out a video of the guys at doppelagent.de experiencing this first hand, although you will want to try it out for yourself.

Okay, so it’s not quite Minority Report level computing. However, with the live human 3D interaction inside of a virtual, yet real, space, all done in the comfort of your browser using Adobe Flash… this is quite amazing, nonetheless. I would love to see this kind of technology take off and be available in a browser… maybe we’re not far off.

fudnik Flex, Technology 3D, Adobe Flash, flash, GE, minority report, smart grid

It is quite common to find yourself with a heinous JavaScript error on a page that until recently seemed to work flawlessly. Perhaps you changed your JavaScript. Maybe you included a 3rd party script or a script from another domain onto your page. Now, everything that was once peachy has turned to sour grapes!

More than likely the problem is that with all the varying scripts on the page, variables from other functions will conflict with variables in the existing code, causing failures and errors, and even worse, overwrite variable values without any notification. It can take hours to track down variables that conflict between scripts before it finally works. Some developers figure that these kinds of issues are probably just inherent in client-side web development and use that as another “reason” as to why JavaScript is inferior.

I don’t think client-side development should be looked at as inherently quirky. Sure there are some browser nuances and environment issues that you can’t control, but you can develop very robust code that works well and adequately serves its purpose.

There are a few key things that you can do to make sure you code is clean and runs in it’s own scope.

First, anytime you create a variable in a function that should not be available outside of the function use the “var” identifier to initialize the variable and restrict it to the local scope.

Second, I would recommend putting much of your code into JavaScript prototype objects. The prototype method of creating objects is JavaScript’s way of creating a class-like object (although prototypes are quite different from actual classes, read up on JavaScript prototypes for more info). In short they allow you to create a group of related function that can share assets between prototype functions (method equivalents).

Later in your code you can instantiate one or more of the prototype classes, each having their own scope and assets. This will keep them from conflicting with any other code.

If you abstract your code well enough, using the power of JavaScript in this manner allows you to create very reusable code that can be used in any application with any combination of JavaScript without problems. There are other things you can do to abstract your JavaScript and make it more functional, but these examples serve the purpose of resolving scope issues and get you on the road to cleaner, reusable code.

fudnik Development, JavaScript ajax, classes, JavaScript, namespace, objects, prototype, variable scope