My Thesis Research

A lot of people ask me to describe my thesis research.  I used to give a complicated answer about using covariance matrices to perform a Karhunen-Loève transform to blah blah blah, but now I just say, “I clean cosmic data.”  Today I created a graphic that illustrates the essence of what I’m trying to do.  (If you don’t see the image, click here.)

Notice how both the signal and noise have “structure”?  My research attempts to uncover those structures and use them to eliminate the noise.  The problem is that taking away noise also takes away signal.  So we need a way to “fill in the gaps.”

Now imagine doing this not for images, but for MASSIVE data sets…and you don’t get to know what the letters are beforehand. Solving this problem to high accuracy is a challenge.

Do “Smart Marks” Help or Hurt the Wrestling Business?

This post was inspired by a question that was asked on Quora.

In some respects smart marks help the business and in others they hurt it. Their best quality is passion. They love wrestling and continuously invest their time and money to support it at all levels. They often comprise a majority of the audience at indy shows, so much so that many smaller promotions might not exist without them. These shows are critical to the business since they provide a stage upon which young wrestlers can hone their craft.

Smart marks can often be assumed as “a given.” Whether the business is hot or cold, they will always be there. On one hand, their constant baseline level of support prevents the floor from falling out of the pro wrestling business. On the other hand, this means they can be more easily taken advantage of since companies like WWE know they will buy whatever they put out.

However, there are some smart marks whose behavior can detract from the quality of a show. Some become a distraction

when they act as if they’re part of the event. For example, a handful with a vendetta against a performer can chant “boring” loud enough to ruin the experience for others.

They can also become jaded to the extent that nothing pleases them anymore. When the same fans attend a company’s shows over and over again, as with the TNA Impact Zone, their reactions die out because they’ve seen everything before. Once a company breaks free of them, as TNA did by leaving the Impact Zone, the shows can become more vibrant.

Other smart marks believe that good wrestling depends on how many spots wrestlers can work into a match. They think that if a wrester can’t perform a flipping powerbomb into a twisting springboard DDT 5 seconds into the match, he’s garbage. This mentality diminishes the impact of all moves and makes them mean less. (As a counterexample, see CM Punk’s piledriver on John Cena during their #1 Contender’s match for the WWE Championship at Wrestlemania 29 on February 25, 2013’s episode of Raw.)

I once heard a story about two ROH wrestlers working a frenetic style during a show. When they came backstage they encountered George South who suggested that they slow down and make individual moves matter more. The wrestlers said they couldn’t do that because the fans would chew them up for it. In effect, the reactions of a small group of fans had directly influenced (arguably negatively) the styles of wrestlers in the nation’s third largest promotion.

So with smart marks you get a mixed bag. In a sense, they are a critical component of the audience that supports young wrestlers and is usually willing to part with their money to support wrestling. At their worst, they can think that what they like matters more than anything else and any wrestler who doesn’t meet their conception of greatness isn’t worth the time of day.

FOIA – We Are Making Progress

This is final part of a 5 part series on the government’s silence of silence and the Freedom of Information Act (FOIA).  Parts 1 through 4 can and should be read first:

Part 1: The Kingston Disaster
Part 2: The Government’s Silence of Science
Part 3: Freedom of Information Act to the Rescue?
Part 4: The Obama Failure

In brief, these articles describe how scientific research gathered by the United States government is often withheld from the general public, a type of action that can quite literally put lives at risk.  The Freedom of Information Act (FOIA) was passed to allow public access to these records, but both the George W. Bush and Obama administrations have so far failed to live up to the promise of the act.

But while there have been substantial challenges with gaining access to important public information, it’s not all doom and gloom.  The fact that we actually have a Freedom of Information Act with an appeals process and judicial review is significant.  The Act continues to have strong support in the NGO community.  A FOIAonline portal has been built with the goal of eventually becoming a one-stop shop for public information.  The Obama administration has taken a strong positive step at Data.gov to “increase public access to high value, machine readable datasets generated by the Executive Branch of the Federal Government.”  This initiative has already saved on infrastructure costs.

And we have had disclosure successes.  In 2008 the United States improved the Consumer Product Safety Act and created a searchable database for consumer information.  The National Oceanic and Atmospheric Administration’s (NOAA) National Climatic Data Center and EPA have done an admirable job of reporting on historical climate variables like temperature, precipitation and drought.  The US Embassy in Beijing has made electronic reports of air quality public when the Chinese government refused to do so.  The federal ENERGY STAR program labels the energy footprint of appliances to aid consumers in making more energy efficient purchases.

Inside federal agencies, it would appear that some progress is being made.  In 2013 UCS released a report entitled Grading Government Transparency in which they examined the ability of scientists at federal agencies to speak freely about their work.  They found that many agencies’ media policies “have shown significant improvement since 2008.”  In particular they note that scientists can now more easily apply their right to express personal views provided they make clear that they are not speaking for their agency.

This right was made considerably easier to exercise when on November 13, 2012, after an arduous 14 year journey, Congress unanimously passed the Whistleblower Protection Enhancement Act.  This act, for the first time, provides specific legal protection to scientists and other federal employees who expose censorship or suppression of federal research.  According to Celia Wexler of the Union for Concerned Scientists (UCS), “We hope that this law will begin a process to change the culture of federal agencies when it comes to whistleblowers. People who protect the public from unsafe drugs, tainted food, defective products, and environmental hazards should not fear for their jobs when they speak up for safety and scientific integrity.”

Since then, other steps have been taken to make it easier for the public to obtain government information.  On May 9, 2013 President Obama issued an executive order making open and machine readable data the new default for government information.  Citing examples like weather data and the Global Positioning System (GPS), the president argued that making federal data freely available “can help fuel entrepreneurship, innovation, and scientific discovery – all of which improve Americans’ lives.”

Then, on February 25, 2014 the US House of Representatives unanimously passed the FOIA Oversight and Implementation Act.  This amendment to the Freedom of Information Act would create a single, free website from which all FOIA requests could be made.  When requests are granted, federal agencies would have to release the information in an electronic and publicly accessible format.  When requests are denied, the appeals process would be streamlined.  The amendment also forces federal agencies to take greater responsibility for their FOIA obligations.

As we see, the system can work.  But there will always be disagreements between the public and federal agencies regarding which information should be disclosed through FOIA and which should be withheld for security reasons.  When public actors feel their claims have been rejected unjustly, they can always consider seeking subpoenas.

Absent that, there are other options at their disposal to extract greater value out of the information that is public.  Private technology companies can offer tools for the sharing and analysis of data.  Librarians can play a more prominent role in gathering and organizing documents.

When the information being disseminated is incorrect, knowledgeable scientists should take action.  They can start issue blogs and connect with members of the media.  Local groups like city councils rarely hear from scientists, so researchers can have an outsized impact in regional issues.  As members of one of the most respected professions, scientists would do well to build relationships with congressional representatives or their science staffers.  Failure to act means allowing dissembling voices fill the vacuum.

With respect to government disclosure, as with most things, the situation is neither entirely good nor bad.  But it is hard to deny that at times we Americans live in a perverse, ironic ecosystem – one in which taxpayers fund government research designed to inform and protect, only to have that same government deny us the results and claim it’s for our protection.  We must continue to hold our government accountable, push for transparency where appropriate and never yield to private interests who would use our ignorance against us.

The Obama Failure

This is Part 4 of a 5 part series on the government’s silence of silence and the Freedom of Information Act (FOIA). Parts 1, 2 and 3 can and should be read first:

Part 1: The Kingston Disaster
Part 2: The Government’s Silence of Science
Part 3: Freedom of Information Act to the Rescue?

In brief, these articles describe how scientific research gathered by the United States government is often withheld from the general public, a type of action that can quite literally put lives at risk. The Freedom of Information Act (FOIA) was passed to allow public access to these records, but we discovered that for a number of reasons, the George W. Bush administration was overly eager to deny such requests.

Many of those critical of the Bush administration’s handling of FOIA requests hoped that the situation would improve under the Obama administration. In fact, one of the new President’s first actions in office was to issue the following instruction, essentially reversing the Ashcroft Memo:

All agencies should adopt a presumption in favor of disclosure, in order to renew their commitment to the principles embodied in FOIA, and to usher in a new era of open Government.

This memo was part of Obama’s Open Government Initiative, “committed to creating an unprecedented level of openness in Government.” Yet surprisingly, government transparency barely improved from the Bush administration and, according to some journalists, got worse. A full 30% gave him a grade of poor to very poor. Recently, OpenTheGovernment.org released an assessment saying that many sophisticated users of FOIA remain tremendously disappointed with the law’s implementation.

There are between 600,000 and 650,000 FOIA requests per year. While less busy agencies can respond within a few weeks, larger agencies like the Department of Defense are flooded with more requests than they have the resources to meet. In these cases, FOIA compliance has effectively become an unfunded mandate. The situation is worsened by the fact that many agency employees are overworked, undertrained, and generally unclear of their obligations under FOIA.

A Bloomberg News investigation last year set out to test the quality of current FOIA compliance. A team of reporters submitted the same FOIA request, for the travel expenses of top agency officials, to 57 agencies. 19 of 20 cabinet-level agencies did not comply within the mandated 20 day window. Even “well past that legal deadline,” about half of the agencies had still not fulfilled the request.

In some cases information is available, but in a form that mitigates its usefulness. Despite Obama’s promise of an online data repository, many information requests still need to be made in person. A significant number of records remain incomplete or redacted. Often data is not in a convenient format like PDFs or tables. Nonuniformity abounds. At the SEC there are different record systems in every department.

There is no uniform method to submit a FOIA request. Some agencies accept submissions by e-mail and others by fax. Some ask the user to complete a web form. Requests to the IRS must actually be sent by post.

Despite President Obama’s vow to “restore science to its rightful place,” scientists who wish to reach out to the public about their research findings were routinely prohibited by public affairs, removing the power of interpretation from data that rarely speaks for itself. They are often denied the right to review, prior to publication, the final versions of reports to which their names are attached or to which their research contributed. Even their ability to obtain access to drafts and revisions of such reports is limited.

The need for scientists to comment on their research is exemplified in the case of the Safe Water Drinking Act. Even though this act requires water utilities to “directly” issue customers water quality reports, the reports are often so technical as to be practically useless. (A water utility proposal to only issue the reports online would further disenfranchise those without Internet access.)

Even if adequate information is ultimately disclosed, delays can mitigate its usefulness. An environmental assessment of TransCanada’s controversial Keystone XL pipeline was criticized by many as giving insufficient consideration to its effects on the climate. The report’s integrity was further compromised when it was discovered that the authors had not only been previously employed by TransCanada, but had published a similarly positive assessment of a Peruvian liquified natural gas pipeline which has since racked up an abysmal environmental and social track record.

These and other concerns were meant to be addressed during a 45-day public comment period, but the State Department (which commissioned the report and has final say on the pipeline’s approval) declined to release those comments, a practice that is routine at other agencies. A FOIA request was submitted, but when an approval decision is expected in the “near term,” any delay in meeting the request can limit the public’s ability to meaningfully influence the outcome.

In fifth and final part of this series I describe how it’s not all doom and gloom! I will outline some of FOIA’s successes as well as highlight improvements that offer hope for the future.

Freedom of Information Act to the Rescue?

This is Part 3 of a 5-part series on the government’s silence of silence and the Freedom of Information Act (FOIA).  Parts 1 and 2 should be read first and can be found here:

Part 1: The Kingston Disaster
Part 2: The Government’s Silence of Science

In brief, these articles describe the circumstances surrounding the rupturing of a coal fly ash containment pond in Roane County, Tennessee.  Government sponsored research that reported the health and environmental risks of such ponds was buried, redacted or otherwise hidden from public view.

Problems such as these were meant to be addressed by the Freedom of Information Act (FOIA).  Enacted in 1966, FOIA grants the public the legal right (also referred to as sunshine laws) to request information from the federal government.  It “provides that any person has a right, enforceable in court, to obtain access to federal agency records, except to the extent that such records (or portions of them) are protected from public disclosure.”

The spirit of FOIA embodies the essence of our American democracy.  We hold that a representative government by the people can, through its collective capacity, understand and prescribe solutions to threats against us.  We hold that a representative government for the people will utilize such knowledge for the security of its citizenry.  We hold that a representative government of the people will be served by the sacred trust we bestow upon our elected leaders.

Instead, we find that our government often defaults to the interests of a select few, frequently under the guise of security.  The precedent was codified in the 2001 Ashcroft Memo in which the then-Attorney General reassured agencies that their deliberations would remain confidential so long as they were “safeguarding our national security, enhancing the effectiveness of our law enforcement agencies, protecting sensitive business information and, not least, preserving personal privacy.”

Attorney General  Ashcroft concluded to the agencies, “When you carefully consider FOIA requests and decide to withhold records, in whole or in part, you can be assured that the Department of Justice will defend your decisions unless they lack a sound legal basis.”

Of course, there are many sensitive issues for which government secrecy is in the national interest.  But the Ashcroft Memo established a sweeping protection for agencies to deny data from the general public as long as they could make some argument about how disclosure would jeopardize law enforcement effectiveness, security, business or privacy.  Given that nearly every issue of import touches at least one of these four categories, FOIA requirements could essentially be ignored at the government’s discretion.

The Bush administration took full advantage of this latitude.  When a 2004 EPA study recommended that hydrofracking fluids, which are injected into the ground during the shale gas extraction process, be regulated under the 1974 Safe Drinking Water Act, then-Vice President Cheney intervened.  Using the business provision of the Ashcroft memo, Cheney had the study redacted by claiming it revealed “trade secrets.”

This secrecy has consequences.  When leaks and spills contaminate local streams and water supplies, scientists are limited in assessing the impacts.  Without knowledge of leaks’ chemical compositions regulation is difficult to justify and contamination is hard, if not impossible, to detect.  All of this serves to reduce the gas industry’s accountability for harms it might cause.  This attendant ambiguity made it easier to pass a provision in the 2005 Energy Policy Act that explicitly exempted fracking fluids from the Safe Water Drinking Act.

It is easy to imagine other circumstances in which the “trade secrets” clause could prove dangerous.  If a train, truck or barge carrying hazardous, but classified, materials were to crash, the secrecy exemption could put first responders in grave risk.

Sometimes, the government decides that even admitting records exist will damage national security or lead to stigmatization.  This justifies the so-called “Glomar response” which allows agencies “to neither confirm nor deny” (read: ignore) FOIA requests.  The Department of Justice, the agency responsible for FOIA enforcement, has broadly supported this right on numerous occasions.

While about 70 countries have their own forms of FOIA, many are plagued by similar issues.  Ireland allows easier access to documents, but many remain unsigned which reduces accountability.  Israel does have an appeals process, but such a request can take years and there are no real penalties for non-compliance.  Even in the European Union, which tends to be more open, the scope of the right remains unclear partly because of the governments’ unwillingness or outright failure to clarify the issue.

In part 4 of this series, we will examine how the widespread hope offered by President Obama’s Open Government Initiative has largely gone unmet.

Say Goodbye to Jade Rabbit

Click on the image below to be transported into the footsteps of the ill-fated Chinese lunar rover, Jade Rabbit. On January 16, 2014, Jade Rabbit (known in Chinese as “Yutu”) completed an examination of the lunar soil. Nine days later, near the end of its second lunar day, China announced the rover had undergone a “mechanical control abnormality” due to complications caused by the “complicated lunar surface environment.” Now I’m no moon expert, but it seems to me that withstanding a lunar environment really needed to be one of Jade Rabbit’s core competencies.

While Jade Rabbit is able to communicate as of February 13, “it still suffers a mechanical control abnormality,” putting to bed its plan to explore the Moon’s Mare Imbrium. Its hibernations during the two-week long lunar nights will become increasingly irrelevant. But we still thank CNSA, Chinanews, Kennth Kremer & Marco Di Lorenzo for the image.

The panorama imaged below was taken from the Chang’e 3 lander.

Pano Button - Yutu Panorama

Tamil Nadu’s Top Station

At the boundary of the Indian states of Kerala and Tamil Nadu lies Top Station, an idyllic hill location that stands at the highest point of the Munnar-Kodaikanal Road. Top Station is about one mile above sea level. On clear days visitors are afforded wonderful views of the surrounding countryside. On days like the one depicted in this video, clouds roll over the mountains providing the calm sensation of being in a cloud forest.

[youtube oNpOrqXmxyY 500 280]

 

Good Morning, Windermere

Continuing with the theme of my travels through India, today I present an interactive panorama from the state of Kerala.  The images that comprise this panorama were taken at the Windermere Estate near Munnar, Kerala at sunrise.  Here the sun rises over the eastern mountains while the hills cascade downward to the south, disappearing into the early morning haze.

Pano Button - Windermere

 

Here’s the location:

[iframe width=”100%” height=”480″ src=”https://maps.google.com/maps?q=10.0551,77.0682&hl=en&sll=10.0551,77.0682&sspn=0.004483,0.00603&t=h&z=17&output=embed”]

 

To see the full India photo gallery, click here.