Japan says it will not negotiate with North Korea unless the country honours previous agreements.
Foreign Minister Seiji Maehara called negotiations "impossible" Monday night, after North Korea hit a South Korean island with an artillery barrage last week, escalating tensions in the regions to heights unseen in years.
China, North Korea's only major ally, has offered to help broker talks among regional powers, but it is unlikely that Seoul will take up the offer.
Tokyo had offered its support to Seoul and Washington to help counter what it called "reckless" acts by the North.
Earlier, South Korea cancelled plans to hold joint military exercises with the U.S. -- operations that North Korea had called a "grave military provocation."
Seoul had planned live-fire artillery drills on the island that was shelled in a North Korean attack that killed four people last week. The attack triggered return fire from the South.
The live-fire exercises planned for Tuesday with a U.S. warship off Yeonpyeong Island were described as "joint war drills" by Pyongyang and would have further heightened the risk of all-out war.
Seoul said the postponement of the drills had nothing to do with Pyongyang's warnings, but rather that they had been announced prematurely.
The North does not recognize a borderline drawn by the United Nations in 1953 when the two countries reached an armistice. Pyongyang's version of the border has Yeonpyeong within North Korea territory.
As a result, several deadly confrontations have taken place in the area in the past decade.
Yeonpyeong Island, which is normally home to about 1,300 people, most of whom live in a fishing village, has now been declared a special security area and only about 300 people remained as of Monday.
South Korea has brought in heavy artillery including long range guns and K-9 howitzer rockers, as well as extra troops to bolster the 3,000 already there, according to reports.
South Korean President Lee Myung-bak said Monday that Pyongyang's actions were unacceptable and warned against further provocation.
"Only a few meters away from where shells landed, there is a school where classes were going on," Lee said. "I am outraged by the ruthlessness of the North Korean regime, which is even indifferent to the lives of little children."
"If the North commits any additional provocations against the South, we will make sure that it pays a dear price without fail."
Andrew Salmon, a correspondent working in the region, said South Korea has been talking tough since the attack, but Seoul's primary goal is to reduce tensions.
If another attack occurs in the near future, however, the South will have little choice but to respond in kind and send a message that such attacks are not acceptable, he told CTV News Channel.
"The South Koreans are almost duty-bound to respond with much more significant force than we've seen in the past, so I think the next instance is going to be far, far more dangerous than this one," Salmon said.
Non-artillery operations between the U.S. and South Korea took place Monday morning, with a U.S. supercarrier and South Korean destroyer doing joint military exercises south of Yeonpyeong Island.
