Learning Statistics Using R: Role Type Classification: Case III (5 of 5)

Now we learn to create a labeled scatterplot using R. In labeled scatterplot we indicate different subgroups or categories within the data on the plot; by labeling each subgroup differently.

Note till now we were using "plot" or "scatter.smooth" function to create our scatter plot, but I have not found a nice and easy method of creating labeled scatterplot using these functions. So instead of these functions we will use another function from "car" package/library.

Recall the hot dog example from case I, in which 54 major hot dog brands were examined. In this study both the calorie content and the sodium level of each brand was recorded, as well as the type of hot dog: beef, poultry, and meat (mostly pork and beef, but up to 15% poultry meat). In this example we will explore the relationship between the sodium level and calorie content of hot dogs, and use the three different types of hot dogs to create a labeled scatterplot.
Lets do this with R using scatter.smooth function

# hotdog.txt is same as TA01_009.TXT
> hotdogdata <- read.table("hotdog.txt", sep="\t", header=T)
> attach(hotdogdata)
> names(hotdogdata)
[1] "HotDog"   "Calories" "Sodium"  
# lets try firt with scatter.smooth function.
# As of version R version 2.6.2 (2008-02-08), 'quanlity' argument
# does not result in a warning but newer version does; pointed out by Gabriele Righetti.
> png("/tmp/hotdogtry1.png", quality=100, width=480)
> scatter.smooth(Sodium, Calories)
# We try added label to this plot using text command/function.
# Bug-Fix: Gabriele Righetti
> text(Sodium,Calories, HotDog)
> dev.off()

So here is our image, but this is not what we wanted. Here we have tried to add text/label to this using "text" function but it has become too ugly to look at. So now lets try creating a better labeled scatter plot using "car" library.
Lets do this with R using "car" library

# we already have data that is attached. so let first load the
# library.
> library("car")
> png("/tmp/hotdogtry2.png", quality=100, width=480)
> scatterplot(Calories ~ Sodium | HotDog)
> dev.off()

So this is out second graph and this looks much better than our earlier. Also there is an entire moive explaining concept of labeled scatterplot at the course website.

Lets Summarize:

• The relationship between two quantitative variables is visually displayed using the scatterplot, where each point represents an individual. We always plot the explanatory variable on the horizontal, X-axis, and the response variable on the vertical, Y-axis.
• When we explore a relationship using the scatterplot we should describe the overall pattern of the relationship and any deviations from that pattern. To describe the overall pattern consider the direction, form and strength of the relationship. Assessing the strength could be problematic.
• Adding labels to the scatterplot, indicating different groups or categories within the data, might help us get more insight about the relationship we are exploring.

Learning Statistics Using R: Role Type Classification: Case III (4 of 5)

To remind; we are learning to examine relationship between a response and a exaplanatory variable. In our case III both variables are quantitative (numerical).

We looked at the method to examine case III relationship between two quantitative variables in previous post. Lets understand it better with few more examples.

Example: The average gestation period, or time of pregnancy, of an animal is closely related to its longevity (the length of its lifespan.) Data on the average gestation period and longevity (in captivity) of 40 different species of animals have been examined, with the purpose of examining how the gestation period of an animal is related to (or can be predicted from) its longevity. (Reference: Rossman and Chance, Workshop Statistics. Discovery with Data and Minitab (2001). Original source: The 1993 World Almanac and Book of Facts).

The actual dataset for this example is available here and also as single zip file that includes all dataset of this site.
Lets examine this dataset using R:

# Lets read the dataset animals.dat into R.
# dataset is separated by a tab (\t) and header row 
# is present but not very intitutive so we will 
# change it later.

> animals <- read.table("animals.dat", sep="\t", header=TRUE)
# print names of the rows.
> names(animals)
[1] "animal"   "gestati3" "longevi4"
# we want to change "gestati3" and "logevi4"
> names(animals) <- c("animal", "gestation", "longevity")
> names(animals)
[1] "animal"    "gestation" "longevity"
# so assigning values to names() changes the names.
# now lets draw the scatterplot and examine the dataset.
# we want to create scatterplot in a png file for upload.
> png("/tmp/animal.png", quanlity=100, width=480)
> scatter.smooth(animals$long, main="Lifespan and Pregnancy", 
  xlab="Longevity (Years)", ylab="Gestation (Days)")
# following writes the graph to file. 
> dev.off()

• Direction: The direction of relationship is essentially positive, that is longer lifespan tends to have have longer times of pregnancy.
• Form: Again the form of relationship (between response and explanatory) variable is linear.
• Outliers: There seems to be one outlier at around 40 years. Lets use R to find out which observation is this?

  # we search for more than 35 year, just to be careful.
  > which(animals$longevity > 35)
  [1] 15
  # 'which' provides us with the observation number that has 
  # logevity > 35. Lets display that observation.
  # combination of which along with dataset give following:
  > animals [which(animals$longevity > 35), ]
       animal gestation longevity
  15 elephant       645        40
  

  
  So our outlier is an observation for elephant. Note that while this outlier definitely deviates from the rest of the data in term of its magnitude, it does follow the direction of the data.

Comments from course site: Another feature of the scatterplot that is worthwhile observing is how the variation in Gestation increases as Longevity increases. Note that gestation period for animal who live 5 years ranges from about 30 days up to about 120 days. On the other hand the gestation period of animals who live 12 years varies much more, and ranges from about 60 days and up to above 400 days.

Example: As another example, consider the relationship between the average fuel usage (in liters) for driving a fixed distance in a car (100 kilometers), and the speed at which the car drives (in kilometers per hour). (Reference: Moore and McCabe, Introduction to the Practice of Statistics, 2003. Original source: T.N. Lam "Estimating fuel consumption for engine size", Journal of transportation Engineering,111 (1985))

The actual dataset for this example is available here (See chapter 2) and also as single zip file that includes all dataset of this site.
Lets examine this dataset using R:

> sf <- read.table("speedfuel.txt", sep="\t", header=TRUE)
> png("/tmp/speedfuel.png", quanlity=100, width=480)
# check column names..
> scatter.smooth(sf$Speed, sf$Fuel, main="",   xlab="Speed (km/h)", 
  ylab="Fuel Used (liters/100km)")
> dev.off()

The data describe a relationship that decreases and then increases - the amount of fuel consumed decreases rapidly to a minimum for a car driving 60 kilometers per hour, and then increases gradually for speeds exceeding 60 kilometers per hour. This suggests that the speed at which a car economizes on fuel the most is about 60 km/h. This forms a curvilinear relationship which seems to be very strong, as the observations seem to perfectly fit the curve. Finally, there do not appear to be any outliers.

Example: Another example and scatterplot taken from course website provides a great opportunity for interpretation of the form of the relationship in context. The example examines how the percentage of participants who completed a survey is affected by the monetary incentive that researchers promised to participants. Here, is the scatterplot which displays the relationship: The positive relationship definitely makes sense in context, but what is the interpretation of the curvilinear form in the context of the problem? How can we explain (in context) the fact that the relationship seems at first to be increasing very rapidly, but then slows down?

Note that when the monetary incentive increases from $0 to $10, the percentage of returned surveys increases sharply - an increase of 27% (from 16% to 43%). However, the same increase of $10 from $30 to $40 doesn't show the same dramatic increase on the percentage of returned surveys - an increase of only 3% (from 54% to 57%). The form displays the phenomenon of "diminishing returns" - a return rate that after a certain point fails to increase proportionately to additional outlays of investment. $10 is worth more to people relative to $0 than to $30.

Learning Statistics Using R: Role Type Classification: Case III (3 of 5)

In next few posts we try to understand direction, form, strength and outliers of our Sign Legibility Distance Vs. Age scatterplot.

We try with simple scatterplot and then try to find its direction using R. So this is how our scatterplot looks when drawn using R.

# read the signdistance datafile that is included in here
# http://krishnadagli.blogspot.com/2008/07/learning-statistics-using-r-data-sets.html

> sigdist <- read.table("signdistance.txt", header=T, sep='\t')
# we want to create scatterplot in a png file, that is easy to upload.
> png("/tmp/signdist.png", quality=100, width=480)
> plot(sigdist, main="Sign Distance", xlab="Driver Age(years)", ylab="Sign Legibility Distance(feet)")
> dev.off()

So this how our scatterplot looks, we try to first find the direction of this plot.
Find direction: (FIXME: Is this correct, what alternatives?): For finding direction of scatter plot we use one of R's function 'scatter.smooth'. The help page says that it plot and add a smooth curve computed by 'loess' to a scatter plot, and Wiki entry here mentions loess, or locally weighted scatterplot smoothing, as one of many "modern" modeling methods. Here is the R code to add a curve to our plot.

# we have already read data in sigdist object, lets use it.
# As we want to create and save the plot, we use png function.
>png("/tmp/sigdistsmooth.png", width=480, quality=100)
> scatter.smooth(sigdist, main="Sign Distance Smooth", 
  xlab="Drive Age(years)", ylab="Sign Legibility Distance(feet)")

Here is how our scatterplot along with a curve looks.
We can see that the direction of the relationship is negative, which makes sense in context since as you get older your eyesight weakens, and in particular older drivers tend to be able to read signs only at lesser distances. (How do we get an arrow drawn over the scatterplot just like its shown in the course?)

Find form: The form of the relationship seems to be linear. Notice how the points tend to be scattered about the line. Although, as we mentioned earlier, it is problematic to assess the strength without a numerical measure, the relationship appears to be moderately strong, as the data is fairly tightly scattered about the line. Finally, all the data points seem to "obey" the pattern- there do not appear to be any outliers.

Learning Statistics Using R: Role Type Classification: Case III (2 of 5)

As we have seen in our previous scatterplot, it is always the case that exaplanatory variable is plotted on horizontal, X-axis and response variable on Y-axis. If, at times we are not able to clearly identify explanatory and response variables then each of them can be plotted on either axis.

Interperting scatterplot: In our case-I we did comparative box plot and in case-II we did comparative bar plot/histogram but now how do we interpret scatterplot? What we dis is to describe the overall pattern of the distribution (of response variable) and any deviations (outliers) from that pattern, we take same approach for scatterplot. That is we describe overall pattern by looking at distribution's "Direction", "Form", and, "Strength" , and along with this we find outliers. Following from the course site puts it in a nice figure.


Lets discuss each of these three in details that describes overall pattern of relationship.

1. Direction: The direction of the relationship can be positive, negative, or neither. We identify the direction of relationship by looking at how scatterplot's points are moving along with x-y plane. Following figures shows example of positive, negative, and neither directions.
   • Positive direction: A positive (or increasing) relationship means that an increase in one of the variables is associated with an increase in the other.
     
   • Negative direction: A negative (or decreasing) relationship means that an increase in one of the variables is associated with a decrease in the other.
     
   • Neither: Not all relationships can be classified as either positive or negative.
2. Form: The form of the relationship is its general shape. When identifying the form, we try to find the simplest way to describe the shape of the scatterplot. There are many possible forms. Here are a couple that are quite common:
   • Linear Form: Relationships with a linear form are most simply described as points scattered about a line:
   • Curvilinear Form: Relationships with a curvilinear form are most simply described as points dispersed around the same curved line:
   • Other Forms: There are many other possible forms for the relationship between two quantitative variables, but linear and curvilinear forms are quite common and easy to identify. Another form-related pattern that we should be aware of is clusters in the data:
3. Strength: The strength of the relationship is determined by how closely the data follow the form of the relationship. Let's look, for example, at the following two scatterplots displaying a positive, linear relationship:
   The strength of the relationship is determined by how closely the data points follow the form. We can see that in the top scatterplot the the data points follow the linear patter quite closely. This is an example of a strong relationship. In the bottom scatterplot the points also follow the linear pattern but much less closely, and therefore we can say that the relationship is weaker. In general, though, assessing the strength of a relationship just by looking at the scatterplot is quite problematic, and we need a numerical measure to help us with that. We will discuss this later in this section.
4. Outliers: Data points that deviate from the pattern of the relationship are called outliers. We will see several examples of outliers during this section. Two outliers are illustrated in the scatterplot below:

Learning Statistics Using R: Role Type Classification: Case III (1 of 5)

Lets start with our next example; understanding relationship between two quantitative variables, ie. both explanatory and response variables are quantitative.

As in previous two cases we compared distribution of response variable with that of explanatory variables. To be specific, in case I we compared distribution of quantitative response with categorical explanatory variable and in case II we compared distribution of categorical response variable with categorical explanatory variable.
Now we have both variables as quantitative, more importantly we have a explanatory variable that is quantitiave. We will start understanding their relationship using "scatterplot".

We start with an example taken from course site.
A Pennsylvania research firm conducted a study in which 30 drivers (of ages 18 to 82 years old) were sampled and for each one the maximum distance at which he/she could read a newly designed sign was determined. The goal of this study was to explore the relationship between driver's age and the maximum distance at which signs were legible, and then use the study's findings to improve safety for older drivers. (Reference: Utts and Heckard, Mind on Statistics (2002). Originally source: Data collected by Last Resource, Inc, Bellfonte, PA.)

Since the purpose of this study is to explore the effect of age on maximum legibility distance,

• the explanatory variable is Age, and
• the response variable is Distance.

Here is what the raw data look like and its available here.
Note that the data structure is such that for each individual (in this case driver 1....driver 30) we have a pair of value (in this case representing the driver's age and distance). We can therefore think about this data as 30 pairs of values: (18,510), (32,410), (55,420)........(82,360).

The first step in exploring the relationship between driver age and sign legibility distance is to create an appropriate and informative graphical display. The appropriate graphical display for examining the relationship between two quantitative variables is the scatterplot.

Here is how a scatterplot is constructed for our example:

To create a scatterplot, each pair of values is plotted, so that the value of the explanatory variable (X) is plotted on the horizontal axis, and the value of the response variable (Y) is plotted on the vertical axis. In other words, each individual (driver, in our example) appears on the scatterplot as a single point whose x-coordinate is the value of the explanatory for that individual, and the y-coordinate is the value of the response. Following images taken from course website illustrate the same.

As we have data set, lets start doing the same using R.

# Read data using read.csv function, here separator is tab.
# Bug-Fix: Gabriele Righetti 
> signdist <- read.csv ("signdistance.txt", sep="\t", header=T)
> plot(signdist, type="b", main="Sign and Distance", col="blue", xlab="Driver Age", 
ylab="Sign Legibility Distance (Feet)")

The output of above simple command plot draws our scatter plot. Also note our scatter plot is different, that is we have lines along with points. That is possible with simple parameter "type=b" in plot commands. See the help page of plot command to see all types.
Another using "type=p" argument.