starmine AI | On-Demand Datasets for Data Vendors, DaaS Providers and Institutions

Powering context-controlled summarization

What happy customers are saying: (aside from "it's free!")

" Hello, I'm pretty curious and amazed by how powerful your summarization of articles is ! I'd like to try it out on a side project so that's why I'm sending you this e-mail :) " - N. Hardy

" You guys are working on some really incredible stuff. Can't wait to get my hands on some code especially after looking at those links now wow...this is more of what we need in the world, this will change everything :D Cool...you're based in SF would love to meetup and grab a coffee sometime and tell you what I'm working on. " - N. Sherriff

" Great work with the context summarization! May I request a copy of the code base? I won't use for commercial purposes, or anything like that. Just academic insight. " - H. Peterson

" I'm really impressed by how accurate your algorithm is , and I would like to try it out summarizing some news concerning the same topic (in different languages). My goal is to create an algorithm that can merge some texts speaking about the same subject, so that's why I will read your mathematical explanations on the API page. FYI I tried it on French news articles, and it seems to be working almost as good as english. (it may work better on latin-based languages right ?) " -

" I do a lot of hobby or experimental projects using NLP, I'd love a chance to use your system.  Is this something you think you may release as open source? " - T. Lelelu

" Amazing work!  Is there a repository I can have access to?  These are exciting times for aspiring amateur academics like myself.  Public knowledge of this calibre is exactly what humanity needs to overcome the debt of our myriad lapses in good faith.   " - W. Sahatdjian

" Requesting API Access. Really intrigued by the amazing work your team has done around content summarization. Was wondering what is the process for applying for API access. Is there an open source implementation that we can contribute towards?  " - R. Tewari

" That's.... incredible. Wow. " - chrismbarr

" I tried it with the 50 Shades of Grey text (btw, impressive text generation work) and I found the results to be just Ok. However, I then tried it with a new text related to an app description and found the result to be quite impressive, both the summarization and the keywords extraction. Well done! " - carlos_argueta

" Is there somewhere I can use this same software/algorithm to actually summarize other things? I am thoroughly impressed. " - primary_action_items

" Really great work - just read the details. Will you writing an academic paper based on your findings? " - PJ Hampton, PhD Researcher, NLP

" That's actually really cool. Now you've got me looking into these algorithms... " - bcolb

" comparing this algorithm with the one i wrote for wingztv.com (shameless plug) i am really shocked by how much better these guys algo is. although mine was just kinda was hacked up. it just sees sentences with more words in common and ranks then by it, and then i just grab the top 5. this algo looks a lot more fancy O.o probably why it's a lot better lol. " - moridin007



Example 1. without context control
(URLs must start with http)



source: http://www.nytimes.com/2016/03/20/magazine/the-secrets-of-the-wave-pilots.html


JSON API Endpoint: GET

Parameters: URL

Returns:


Example 2. with context control options
Paste any text below
(example texts: 50 Shades of Grey, FOXO3A, Star Wars, Raiders of the Lost Ark)
Summary size (default: 2 sentences)

Context control sensitivity level


Optional Context Control:

In the context of: (e.g. "geography", "money" or "time")
Use this to control the context of a general summary





JSON API Endpoint: POST
Parameters:
Results:


Example 3.
SumBot @SumBot - context-controllable summarization for any URLs right in your favorite slack channel.


A context-controllable content summarization example based on the Raiders of the Lost Ark movie plot:




An example of summarization without context control:

Standard Summarized Content in 2 sentences (source)

Belloq and the Dietrich seize the Ark and Marion but cannot locate Jones who stows away aboard the Nazi U-boat and travels with them to an island in the Aegean Sea... Belloq and Nazi officer Colonel Dietrich arrive seize the Ark from Jones throwing Marion into the Well of Souls with him before sealing it back up...

Auto Generated Ranked Tags

jone, ark, nazi, marion, belloq, headpiece, toht, soul, sallah, dietrich, ravenwood, arrive, staff, inform, agent, army, travel, aboard, escape, tavern, face, digging, forced, boat, suddenly, hand, archaeologist, intelligence, ra, threaten, searching

An example of summarization with context control:

Summarized Content in the context of "sun" (source)

The Nazis know that Ravenwood is the leading expert on the ancient Egyptian city of Tanis and that he possesses the headpiece of the Staff of Ra...A vortex of flame forms above the opened Ark and energy surges out into the gathered Nazi soldiers killing them all...

Auto Extracted Ranked Tags in the context of "sun"

ancient, energy, giant, travel, suddenly, fire, turn, hot, sand, day, burned, melted, remove, emerge, killing, people, face, aegean, ra, wing, government, surrounded, eye, air, egyptian, head, power, jone, ark, nazi, marion

Original Content:

http://en.wikipedia.org/wiki/Raiders_of_the_Lost_Ark#Plot

In 1936, archaeologist Indiana Jones braves an ancient booby-trapped temple in Peru and retrieves a golden idol. He is confronted by rival archaeologist Renéelloq and the indigenous Hovito eople. Surrounded and outnumbered, Jones is forced to surrender the idol to Belloq and escapes aboard a waiting floatplane. Jones returns to his teaching position at Marshall College, where he is interviewed by two Army Intelligence agents. They inform him that the Nazis, who are obsessed with the occult, are searching for his old mentor Abner Ravenwood. The Nazis know that Ravenwood is the leading expert on the ancient Egyptian city of Tanis, and that he possesses the headpiece of the Staff of Ra. Jones deduces that the Nazis are searching for the location of the Ark of the Covenant; the Nazis believe that if they acquire the Ark their armies will become invincible. The Staff of Ra is the key to finding the Well of Souls, a secret chamber in which the Ark is buried. The agents authorize Jones to recover the Ark to prevent the Nazis from obtaining it. He travels to Nepal and discovers that Abner has died, and the headpiece is in the possession of Ravenwood's daughter Marion. Jones visits Marion at her tavern, where she reveals her bitter feelings toward him from a previous romantic affair. She rebuffs his offer to buy the headpiece, and Jones leaves. Shortly after, a group of thugs arrive with their Nazi commander, Arnold Toht. Toht threatens Marion to get the headpiece, and her bar is set on fire when Jones comes back to intervene. Toht severely burns his hand trying to pick up the hot headpiece and flees the tavern screaming. Jones and Marion escape with the headpiece, and Marion decides to accompany Jones in his search for the Ark so he can repay his debt to her. The pair travel to Cairo, where they meet up with Jones's friend Sallah, a skilled excavator. Sallah informs them that Belloq and the Nazis are digging for the Well of Souls with a replica of the headpiece, created from the scar on Toht's hand. They quickly realize the Nazi headpiece is incomplete and that the Nazis are digging in the wrong place. The Nazis kidnap Marion and it appears to Jones that she is killed in an exploding truck. After a confrontation with Belloq in a local bar, Jones and Sallah infiltrate the Nazi dig site and use their staff to correctly locate the Ark. Jones, Sallah, and a small group of diggers unearth the Well of Souls and Jones is forced to face his fear of snakes to acquire the Ark. Belloq and Nazi officer Colonel Dietrich arrive, seize the Ark from Jones, throwing Marion into the Well of Souls with him before sealing it back up. Jones and Marion escape to a local airstrip, where Jones has a brutal fistfight with a Nazi mechanic before blowing up a flying wing. The panicked Nazis remove the Ark in a truck and set off for Berlin, but Jones catches them and retakes it. He makes arrangements to take the Ark to London aboard a tramp steamer. The next day the Nazis arrive and intercept the boat. Belloq and the Dietrich seize the Ark and Marion but cannot locate Jones, who stows away aboard the Nazi U-boat and travels with them to an island in the Aegean Sea. Once there, Belloq plans to test the power of the Ark before presenting it to Hitler. Jones reveals himself and threatens to destroy the Ark with a bazooka, but Belloq calls the bluff and Jones surrenders rather than destroy such an important historical artifact. The Nazis take Jones and Marion to an area where the Ark will be opened and tie them to a post to observe. Belloq performs a ceremonial opening of the Ark, which appears to contain nothing but sand. Suddenly, angelic and beautiful, ghost-like beings emerge from the Ark and float around the assembly. Jones cautions Marion to keep her eyes tightly closed and not to observe what happens next. Belloq and the others look on in astonishment as the apparitions are suddenly revealed to be angels of death. A vortex of flame forms above the opened Ark and energy surges out into the gathered Nazi soldiers, killing them all. As Belloq, Toht and Dietrich all scream in terror, the Ark turns its fury on them: Dietrich's head shrivels up, Toht's face is melted off his skull and Belloq's head explodes. Flames then engulf the remains of the doomed assembly, save for Jones and Marion. The Ark's lid is blasted high into the air before dropping back down onto the Ark and sealing it. Jones and Marion find their ropes burned off and embrace. In Washington, D.C., the Army Intelligence agents inform Jones and Brody that the Ark is someplace safe and will be studied by "top men\". The Ark is shown being permanently stored in a giant government warehouse among countless similar crates. ...


Tools:

  • Dataset Signal Explorer
  • Dataset Builder
  • Google Trends as concepts
  • API issues or quesions? Contact us:

    Custom Data Streams, Features, Real-time, Context-control